Review of the Commodity Pool Operator and Commodity Trading Advisor Disclosure Document Review Program
of
National Futures Association
�
�
Commodity Futures Trading Commission
Division of Trading and Markets
�
September 1999
�
�
This is a report on the Division of Trading and Markets� (�Division�) review of the design and execution of the Commodity Pool Operator ("CPO") and Commodity Trading Advisor ("CTA") Disclosure Document Review Program (�Program�) of National Futures Association (�NFA�). This review is part of the Division's ongoing program of reviews, which have been conducted periodically for many years, to assess the design and overall effectiveness of self-regulatory organization ("SRO") rule enforcement programs.[1] The Division's review focused on the period from May 1, 1998 to April 30, 1999 (the "target period").
The purpose of this review is to examine NFA�s implementation and operation of the Program. This is the first review of the Program since the delegation of this responsibility to NFA. Specifically, the focus of the Division�s review was: (1) to determine whether the Program enables NFA to ensure its CPO and CTA Members' compliance with the disclosure requirements of Part 4 of the Commission�s regulations; and (2) to identify specific areas of concern and, as appropriate, to recommend corrections or enhancements in those areas.
Section II of this report briefly describes the statutory and regulatory background of the Program. Section III outlines the Division�s findings and recommendations. Section IV provides a description of the Program as it is currently implemented, and sets forth in detail the results of the Division�s review.
CPOs and CTAs are required to file disclosure documents with the Commission for the pools and trading programs they offer pursuant to Part 4 of the Commission�s regulations.[2] They are also required to file these documents with NFA.[3] The basic disclosure requirements are intended to ensure that potential investors will be apprised of material facts regarding managed investments and advisors so that they can make an informed decision about a particular investment or advisory service before committing their funds.[4]
Effective November 1, 1997, the Commission delegated to NFA the authority and responsibility to conduct the reviews of disclosure documents required to be filed with the Commission by CPOs of privately offered commodity pools[5] and by CTAs for all offered trading programs.[6] As discussed further below, these functions are performed by the Disclosure Document Review Team (�DDRT�), a group within NFA�s Compliance Department.
������� The Division finds that NFA's Program generally meets the requirements of the CEA, the Commission's regulations, and relevant interpretations. The staff NFA has assigned to the program are experienced and well trained. NFA has established appropriate procedures for conducting disclosure document review, and has conducted reviews in a timely manner. When followed, NFA�s procedures result in effective review of disclosure documents. The Division found, however, that these procedures were not always followed in a number of areas, including steps for disclosure document review, documentation of communications with members, instant filing, and data entry and maintenance.
In conducting the review, the Division analyzed NFA�s accuracy by comparing, for each document, compliance problems documented in NFA�s records against the compliance problems observed by Division staff. Some of the compliance problems which were not documented by NFA staff may have simply been overlooked by DDRT in its reviews. This issue is addressed in Recommendation 1 below, concerning the use of the NFA checklist. Other compliance problems found by Division staff may not have been documented due to differences in views between NFA staff and Division staff concerning the requirements of the Commission�s regulations. This issue is addressed in Recommendation 4 below, concerning enhanced communication between Division staff and NFA staff.
������� The Division therefore recommends that NFA adopt the following changes to its administration of the Program:
Recommendation 1: NFA should revise its procedures to require that staff complete an NFA-developed disclosure document review checklist for each review performed.������������ NFA�s DDRT has developed a checklist to be used as a guide in reviewing disclosure documents. As part of this rule enforcement review, Division staff have reviewed this checklist, and have determined that it includes a comprehensive list of items that are required by the CEA and Commission regulations to be contained in the disclosure document. The checklist includes clear and complete descriptions of each of these items.
������� NFA staff disclosed in interviews, however, that members of the DDRT have discretion to forego completing the checklist in conducting their reviews. The Division�s review confirmed that this discretion is frequently exercised. Checklists were completed in only nine out of the 49 cases in the Division�s sample of CPO documents, and in only 12 out of the 49 cases in the Division�s sample of CTA documents. Thus, overall, checklists were completed in only 21% of the cases in the Division�s sample.
������� To be sure, the important issue is whether the DDRT accurately identifies problems with disclosure documents, not whether or not they use a particular means to do so. The Division�s findings indicate that while the DDRT did an excellent job in identifying problems with disclosure documents in cases where the checklist was used, its performance was not as good in cases where the checklist was not used. (These findings are discussed in greater detail in Section IV.C.1. below.) For example, across the entire sample, Division staff identified 70 problems that existed in documents where DDRT staff used the checklist. NFA�s records indicate that DDRT staff documented 61 of these, for an accuracy rate of 87%. Division staff identified 115 problems in documents where DDRT staff did not use the checklist. NFA�s records indicate that DDRT staff documented 62 of these, for an accuracy rate of only 54%. This difference is statistically significant at a point well above the 99% confidence level.[7]
In addition, the use of a checklist appears to significantly increase the likelihood that discrepancies between NFA�s databases and the disclosure documents were found. Of the 14 discrepancies where a checklist was used, ten were caught by NFA (71%). Of the 29 discrepancies where a checklist was not used, only 14 were noted by NFA (48%). This difference is statistically significant at a point above the 85% confidence level. See Section IV.C.6. for a more detailed analysis.
������� Accordingly, the Division recommends that NFA revise its procedures to require that a checklist be completed for all document reviews. NFA may wish to develop more limited checklists for specific types of filings, e.g., instant filings.
�
Recommendation 2: NFA should limit its application of instant filing treatment to documents where such treatment has been specifically requested and where the appropriate representations have been provided.CFTC Advisory 95-44[8] allows CPOs and CTAs to submit certain disclosure documents for �instant� filing review. These documents, which must be substantially similar to documents which have previously been filed, are reviewed in an expedited manner, generally based on an explicit representation in a letter from the CPO or CTA that the document contains no material changes from the prior filings. The purpose of this expedited procedure was to enable staff �to concentrate resources on those filings that most need review.�
Requiring a representation that there have been no material changes from a prior filing provides an important safeguard, ensuring that the document has, at one point, been subject to a more searching review. Consequently, Advisory 95-44 specifies that unless a transmittal letter stating that there have been no material changes from the prior filing is provided, the document will be processed in the ordinary manner.
Among the sample of 98 documents, the Division found that 13 documents (seven CTA and six CPO) were accepted by the DDRT as instant filings without the requisite request or representation. A closer review of these documents revealed that six of these documents (four CTA and two CPO) contained compliance problems, and should not have been accepted.[9] DDRT should have provided comments on these issues and should not have accepted the documents for instant filing.
Although the Division supports the DDRT's goal of completing its reviews of disclosure documents quickly and its efforts to encourage qualifying CPOs and CTAs to use the instant filing procedure, the Division believes that DDRT should not grant instant filing treatment to documents for which instant filing relief has not been requested or where the requirements for instant filing treatment � including the representation that the document has not been materially changed from a prior, accepted document � have not been met.
������� NFA uses its DDRT Database to track and locate documents the DDRT receives, including both disclosure documents and related correspondence between NFA and the registrant. The database is also used to track the DDRT�s performance in responding to filings promptly. As discussed in more detail below, NFA has procedures governing how such documents should be tracked.
������� In the course of its review, the Division noted a significant number of instances where the tracking data found in NFA�s database did not correspond with the information found on the hard copies of documents in NFA�s files. The Division found that many of DDRT�s document tracking procedures were not followed correctly, and were sometimes not followed at all. (See Section IV.C.5.) It appears that the overriding reason for these errors and inconsistencies is that the responsibility for entering tracking data is diffused. The Division suggests that a centralized procedure for entering all data tracking the processing of disclosure documents and related correspondence would provide the control and consistency necessary to maintain accurate and complete records of these filings.
������� The Division notes that NFA has already taken the first step in resolving this problem. In June 1999, a single DDRT Administrator was assigned responsibility for logging into the database documents that arrive at NFA and assigning the documents to individual reviewers. This is an improvement over the prior system, whereby this task would rotate among DDRT staff on a weekly basis. Many of the errors or missing data the Division encountered, however, occurred in connection with tracking correspondence after the initial receipt of the document, when data entry and maintenance was the responsibility of the reviewer assigned to the document. For example, many comment letters and phone comments originating from NFA were not entered into the database at all. In addition, revised documents that responded to comment letters were designated as �New Documents,� even though NFA�s DDRT Procedures Manual states that revisions to previously filed documents should be designated as revisions in the database.
The Division recommends that NFA further centralize the data entry and filing duties within the Program. We are encouraged that NFA recognizes that this is an area of concern, and urge them to continue working on an effective system of internal control that would address these problems.
������� In proposing to assume responsibility for Disclosure Document review, NFA suggested periodic meetings between NFA staff and CFTC staff to discuss emerging disclosure issues. While a few such meetings took place in the months after the turnover, none have been held recently.
�
In the course of this review, both DDRT staff and Division staff have indicated that regular meetings would be beneficial to permit an exchange of views on both mature and emerging disclosure issues and regulatory issues. Such meetings may also serve as a forum for Division and NFA staff to work together to provide guidance to the CTA and CPO community concerning disclosure requirements. Accordingly, the Division recommends that Division staff and DDRT staff work together to meet, perhaps two to three times per year, to discuss such issues.
�
The Commodity Futures Trading Commission Act of 1974, which created the Commission, provided for the establishment of �registered futures associations� as self-regulatory bodies.[10] On September 22, 1981, the Commission approved NFA�s application for registration as a futures association. Currently, NFA is the only registered futures association.
������� Section 17(p) of the CEA requires NFA, as a registered futures association, to adopt rules that require NFA to: (1) establish training standards and proficiency testing for persons for whom it has registration responsibilities, and create a program to audit and enforce compliance with those standards; (2) establish minimum capital, segregation, and other financial requirements and implement a program to audit and enforce compliance with such requirements; (3) establish minimum standards governing the sales practices of its members and associated persons for transactions subject to the provisions of the CEA; and (4) establish supervisory guidelines to protect the public interest relating to the solicitation by telephone of new futures or options accounts.
������� Pursuant to the authority delegated by the Commission to NFA under �17(o) of the CEA,[11] in November 1997, NFA instituted its Program for the review of disclosure documents of CTAs and CPOs of privately offered commodity pools.
������� Commission rules require that CPOs provide prospective participants with a �Disclosure Document� containing specified information before soliciting, accepting or receiving any funds, securities or property from them.[12] Similarly, CTAs are required to provide disclosure documents containing specified information to prospective clients before soliciting or entering into an agreement to direct or to guide the clients commodity interest account or trading.[13] Both CPOs and CTAs must file their disclosure documents with the Commission at least 21 calendar days before they first deliver the documents to prospective clients.[14] In addition, CPOs and CTAs may not use their disclosure documents for more than nine months from the effective dates thereof.[15] Prior to November 1, 1997, Division staff reviewed disclosure documents filed by CPOs and CTAs. When staff noted instances in which a document failed to comply with either the CEA or the Commission�s regulations, they would send the registrant a letter noting such issues, and would work with the registrants to resolve them.[16]
CPOs and CTAs have also been required to file disclosure documents with NFA. NFA�s Compliance Rule 2-13[17] requires that NFA members file with NFA a copy of any document required to be filed with the Commission pursuant to Part 4 of the regulations. Prior to its implementation of the Program, NFA�s review of CPO and CTA disclosure documents was conducted primarily during field audits, to ascertain whether information contained in the documents was consistent with the firm�s operations.[18] In addition, NFA conducted desk reviews[19] for all newly filed disclosure documents and certain other documents.[20]
�
In response to NFA�s longstanding offer to assume responsibility for the disclosure document review program, the Commission requested in May 1997 that NFA provide a detailed proposal setting forth the standards and procedures to be followed with respect to the Program and the records that would be generated to facilitate Commission oversight.[21] After receiving NFA�s proposal,[22] and in light of NFA�s experience in receiving and reviewing disclosure documents of CPOs and CTAs during the course of on-site audits as well as through the desk review program, the Commission delegated to NFA authority to review all disclosure documents filed by CTAs and all disclosure documents filed by CPOs for privately offered commodity pools and to provide registrants with notice of deficiencies.[23]
The Program is currently staffed by an eight-member Disclosure Document Review Team (�DDRT�) within NFA�s Compliance Department.
This review focuses on the operation of NFA's CPO and CTA Disclosure Document Review Program for the period from May 1, 1998 to April 30, 1999. The Division�s review covered the following areas:
(1) Analysis of DDRT�s Written Procedures. Division staff first obtained from NFA copies of the procedures used by NFA's DDRT staff in reviewing disclosure documents to determine whether those procedures are sufficient to ensure that the disclosure documents are consistent with the CEA and the requirements of Part 4 of the Commission's regulations. As noted above, the Division found that the DDRT checklist contains a comprehensive list of items required by the CEA and related Commission regulations, and clear and complete descriptions of those items. (This analysis is discussed in Section IV.C.1 below).
(2) Analysis of DDRT�s Review of the Sample of Disclosure Documents. Division staff selected a sample of CTA and CPO files to examine, and then reviewed each CPO and CTA document in the sample without referring to NFA�s review of the document. The Division developed a comprehensive list of the problems in each document in the sample by comparing the results of the Division�s review to NFA�s record of the document review conducted by DDRT staff. Division staff examined both hardcopy files and the DDRT database for the documents in the sample to determine whether and how NFA communicated issues discovered during the review process and whether acceptance letters were sent in accordance with NFA�s procedures. Based on this review, Division staff developed an analysis of the number and types of issues found in the documents and of the issues identified in the DDRT�s reviews. (This analysis is discussed in Sections IV.C.1 and IV.C.2 below).
(3)� Analysis of DDRT�s Use of Instant Filing Treatment. Division staff also reviewed the sample for all instances where DDRT records indicated that documents were handled as instant filings, pursuant to Division Advisory 95-44. Division staff identified situations where NFA responded to proper requests for instant filing relief, as well as situations where firms were granted instant filing relief although no proper request (or no request at all) was filed. These situations were distinguished based on transmittal letters on file with NFA, letters sent to the registrants and notations made to the database for each case. (This analysis is discussed in Section IV.C.3 below).
(4) Analysis of Timeliness of DDRT�s Review and of NFA�s Tracking System. Division staff reviewed the sample to ascertain the amount of time between the date NFA received a document and the date on which review of that document was completed. In addition, Division staff compared the entries in the DDRT database to timestamps on the hardcopy in the file for each document in the sample to determine whether NFA�s system for tracking the status of disclosure documents was adequate.�� Division staff then recorded any discrepancy between the data entered and the dates contained on the hardcopy or other documentation in the file. (This analysis is discussed in Sections IV.C.4 and IV.C.5 below).
(5) Analysis of DDRT�s Utilization of NFA Databases in Document Review. NFA's procedures indicate that NFA staff are required to check NFA databases for registration information, disciplinary history and compliance records as part of the review process. Division staff compared the disclosure documents in the sample to the NFA databases to determine whether any discrepancies existed between the disclosure document and the databases at the time of DDRT�s review. Staff also reviewed the comment letters and notes of NFA�s calls to registrants to determine whether NFA had brought the discrepancies to the attention of the registrants. (This analysis is discussed in Section IV.C.6 below).
(6) Analysis of Coordination between the DDRT and NFA Audit Staff. Division staff reviewed the sample to determine whether procedures for coordination between the DDRT and NFA audit staff regarding issues arising from disclosure document review were followed. Division staff reviewed audit referrals from DDRT to the audit staff, as well as audits of firms in the sample that were conducted prior to review of their disclosure documents to determine whether audit findings relating to the disclosure documents were addressed. (This analysis is discussed in Section IV.C.7 below).
(7) Analysis of NFA�s Identification of Novel or Complex Issues. Division staff reviewed NFA�s procedures for identifying and addressing novel or complex issues arising from disclosure document review and for determining whether issues should be discussed with Commission staff. Division staff reviewed all documents in the sample to determine if there were any novel and complex issues that were reflected in the documents, and whether NFA followed their procedures with regard to these cases. (This analysis is discussed in Section IV.C.8 below).
(8) Analysis of Training and Experience Levels of NFA DDRT Staff. In order to evaluate the training and experience of the DDRT, Division staff interviewed each member of the DDRT. The interviews covered such areas as degree of education, length of employment with NFA, experience with CPO and CTA audits and disclosure documents, level of training both before and after joining the DDRT, and involvement in industry meetings designed to discuss CPO and CTA disclosure document issues and operations. (This analysis is discussed in Section IV.D below).
In conducting this review, Division staff obtained and reviewed: written policies and procedures (including the �DDRT Procedures Manual� and checklists used by the DDRT in reviewing disclosure documents); a sample of disclosure documents reviewed by NFA staff and the checklists and correspondence related to the disclosure documents in the sample; completed audit examinations; and information on the experience and education of NFA's DDRT staff. In addition, Division staff interviewed NFA managers and staff.
������� NFA provided Division staff with access to NFA's computerized database and tracking system for the Program (�DDRT Database�). Division staff also reviewed information from the following NFA databases:
��������� Registration (Membership Registration Receivables System ("MRRS"))[24]
��������� Compliance (Financial Analysis and Audit Compliance Tracking System ("FACTS"))
��������� Disciplinary information (Clearinghouse of Disciplinary Information)[25]
In order to test the various components of the Program a random sample of 49 CTA filings and 49 CPO filings was drawn from all the documents received by NFA during the target period.[26]
Division staff reviewed each of the CPO and CTA sample documents, before referring to NFA�s review of the document, and documented the compliance problems presented in the document. The results of this review were later compared to the documentation of the review conducted by DDRT staff.[27] The accuracy of DDRT�s review was measured by comparing the set of issues documented in NFA�s files[28] against the set of issues properly identified by either CFTC or NFA reviewers.
������� In analyzing compliance problems, Division staff distinguished �more significant� and �less significant� problems. More significant problems are those for which Division staff would reject a document, such as failure to disclose the firm�s principals or omitting the amount of the fees paid to the pool�s introducing broker. Less significant problems are those items which, individually, would not be sufficient to render the document unacceptable for use for solicitation, but which should nonetheless be corrected at the next update.[29]
NFA documented 60 of 80 compliance problems (75%) found in the sample of CPO documents: Ten of 15 more significant problems (67%) and 50 of 65 less significant problems (77%). NFA documented 63 of 105 compliance problems (60%) found in the sample of CTA documents: Twenty-four of 40 more significant problems (60%) and 39 of 65 less significant problems (60%). The Division identified very few cases in which NFA incorrectly considered an issue to be a compliance problem when it was not.
In analyzing compliance problems that were not documented by DDRT staff, the Division found no substantive areas where DDRT consistently failed to document compliance problems. The Division did observe, however, a significant difference in accuracy depending on whether or not the reviewer completed a DDRT-developed checklist.
The DDRT has developed checklists detailing the information required to be contained in CTA and CPO disclosure documents and specifying certain procedures that should be conducted during the review, such as checking audit reports issued within the past year and disciplinary information regarding the firm�s principals.[30] (Copies of the CTA and CPO checklists are attached as Appendix A and Appendix B, respectively.) The checklists are maintained in computer spreadsheets and are completed on-line. Division staff has reviewed these procedures and has determined that the procedures are sufficient to ensure that the disclosure documents are consistent with the CEA and the requirements of Part 4 of the Commission's regulations, as interpreted by applicable advisories and interpretations.
�������
The DDRT Procedures Manual states that the checklist is to be used as a guide when reviewing disclosure documents, and that:
�When completing the checklist, the reviewer should:
������ Reference to the page number where the required disclosure is included;
������ Document any unusual items or deficiencies noted, either in the cell note for the applicable disclosure requirement or in the notes section;
������ Document any phone conversations between NFA and the firm in the phone log or in the notes section of the checklist.�
DDRT staff disclosed in interviews that, in practice, use of the checklist is optional. The Division found that DDRT reviewers frequently chose to forego filling out the items in the checklist:Of the 49 CPO documents reviewed, 40 did not have a checklist, and of the 49 CTA documents reviewed, 37 did not have a checklist.
The Division analyzed the accuracy of the reviews performed by DDRT with and without checklists. In general, NFA�s performance on those documents for which the checklist was used was uniformly better than on those documents for which the checklist was not used, and those differences were statistically significant at high confidence levels.
For all 77 documents without checklists, NFA correctly identified 14 of 33 more significant problems (42%), while in the 21 documents with checklists, NFA identified 20 of 22 more significant problems (91%). Focusing on less significant problems, NFA identified 48 of 82 (59%) in documents without checklists, and 41 of 48 (85%) in documents with checklists. For all problems, NFA identified 62 of 115 without checklists (54%) and 61 of 70 with checklists (87%). These differences were statistically significant at a confidence level well above 99%.
The Division�s findings regarding DDRT�s identification of more significant and less significant problems, both with and without the checklist, are presented in the table below:
�CTA Documents[31] |
Number of Problems |
Number NFA Identified |
% NFA Identified |
|
More Significant Problems: |
� |
� |
� |
|
No Checklist |
22 |
8 |
36% |
|
With Checklist |
18 |
16 |
89% |
|
Less Significant Problems: |
� |
� |
� |
|
No Checklist |
36 |
12 |
33% |
|
With Checklist |
29 |
27 |
93% |
�CPO Documents |
� |
� |
� |
|
More Significant Problems:[32] |
� |
� |
� |
|
No Checklist |
11 |
6 |
55% |
|
With Checklist |
4 |
4 |
100% |
|
Less Significant Problems:[33] |
� |
� |
� |
|
No Checklist |
46 |
36 |
78% |
|
With Checklist |
19 |
14 |
74% |
�
A more complete statistical analysis of these results is found in Appendix D.
�
������� Although DDRT does not have specific procedures for reviewing documents on a limited basis, staff indicated in interviews that they have the discretion to review updated documents solely with respect to the areas where changes have been made from the prior filing.In 18 of the CPO cases and 26 of the CTA cases, there was a note indicating why a checklist was not completed. In most of these cases, the notes indicated that the filings contained only limited changes, such as supplements, instant filings, and updated filings with minimal changes from the prior filing. As noted in the margin,[34] the Division analyzed the possibility that the explanation for DDRT missing problems in documents without checklists was due to NFA�s review being limited to the areas of the document that were identified as changed. Based on this analysis, the Division concludes that this possibility does not provide a sufficient explanation for the pattern of missed problems in documents where the DDRT reviewer did not complete a checklist.[35]
The DDRT Procedures Manual specifies that, when completing the checklist for a document, any deficiencies found should be documented either in the cell note corresponding to that item or in the notes section of the checklist. DDRT staff indicated in interviews (and the Division�s review confirmed) that deficiencies are sometimes documented instead in the notes or telephone notes sections of the database record corresponding to the document. The Division found this multiplicity of locations where problems may be documented to cause confusion.
According to DDRT staff interviews, the reviewer has discretion regarding whether to convey comments on the document to the registrant by telephone or in writing. In exercising this discretion, the reviewer is expected to take any preference expressed by the registrant into account. Both written and telephonic comments to registrants regarding problems in their disclosure documents should be recorded in the correspondence folder. In some cases, a letter may still be sent even if the comments have been discussed with the registrant by telephone. Written comments are generally transmitted to the registrant in the same manner as the correspondence is received by NFA (e.g., facsimile, electronic filing, or hard copy).
DDRT has standard opening and closing paragraphs for comment letters, but does not have standard language for communicating comments regarding specific issues in the disclosure document. There is no standard script for comments conveyed by telephone.
The Division found that, where comments were communicated in writing, problems with the document appeared to be adequately discussed. The comments provided clear and logical guidance for correcting items that were not in compliance with the regulations, applicable interpretations and advisories. In most cases, however, comments were communicated by telephone rather than in writing. The Division reviewed DDRT�s notes of such conversations. These notes appeared to provide complete records of dates of the conversations, firm personnel with whom the reviewer spoke, descriptions of problems found in the documents, and the manner in which the problems described were resolved.
Upon receipt of a revised disclosure document, DDRT staff check the correspondence file for the existence of open comments. If comments exist, the disclosure document is reviewed to determine whether adequate changes have been made to address those comments. Once any comments are resolved, DDRT sends an acceptance letter. DDRT�s written procedures indicate that acceptance must be communicated in writing (either by mail, fax or electronically), although DDRT staff have indicated that the reviewer may also convey the acceptance by telephone. The written procedures provide that a hard copy of all correspondence must be maintained along with the disclosure document. DDRT staff indicated, however, that hard copies of correspondence are not always maintained. Because acceptance letters may be referred to in Commission or NFA enforcement actions, the Division urges that contemporaneous hard copies of all such letters be printed and maintained by NFA.
The Division found that DDRT sent an acceptance letter for 45 of the 49 CPO filings and 47 of the 49 CTA filings. With respect to the remaining six cases, the Division was unable to find either a copy of an acceptance letter or an indication, in either the tracking system or the registrant�s file, that the filing had been abandoned by the submitter. Due to the problems with the tracking system noted below in Section 5, NFA�s records do not provide sufficient information to determine whether acceptance letters were (or should have been) sent in these six cases.
�
Further, the Division found that the DDRT provided an acceptance letter to a CTA in three instances where the CTA�s registration had not become effective prior to the date of the acceptance letter. One of these three letters properly warned that the disclosure document could not be used for solicitation until the CTA�s registration became effective. The other two such letters indicated that the disclosure document was accepted without conveying this important warning. The Division suggests that any acceptance letter sent in a case where the firm's registration is not yet effective should state that the document may not be used until the firm is registered.
�
The Commission instituted a program in 1995 to allow expedited treatment (referred to as �instant� filing review ) for certain disclosure documents. Pursuant to Advisory 95-44,[36] instant filing review is available to registrants who file amended documents in connection with existing pools and CTA programs, and to registrants who submit filings for new pools that are substantially similar to pool documents previously filed, reviewed and accepted and that contain no material changes from the previously filed document.[37] Documents submitted in accordance with this procedure may be used for solicitation shortly after they have been filed, and the documents are not reviewed as thoroughly as other filings. The Advisory provides that a letter indicating either acceptance or rejection of the filing will generally be sent within two days of receipt of the document.
�
The Advisory requires that CPOs and CTAs seeking instant filing treatment file their documents under cover of a transmittal letter that contains an explicit representation that there have been no material changes from a prior filing, and that specifies whether the current filing is an updated document for an existing pool or a document for a pool being offered for the first time. Changes to the prior document are to be redlined.
Of the Division�s sample, the DDRT reviewed four CTA documents and ten CPO documents where a request for instant filing treatment was properly made and accompanied by the representations required by CFTC Advisory 95-44. Two of these 14 documents contained more significant problems (and two contained less significant problems), but there was no documentation of any comment by NFA concerning these problems. None of these documents was reviewed using a checklist. The Division suggests that NFA develop and regularly use a checklist appropriate for instant filings, as noted in Recommendation 1.
In addition, seven CTA documents and six CPO documents were accepted by the DDRT as instant filings despite the fact that there was no request for such treatment, and no representation that the document was similar to an earlier document.[38] Four of the CTA documents and one of the CPO documents contained more significant issues (and five CTA documents and three CPO documents contained less significant issues) for which there was no documentation of comments by NFA.
The Division recommends that NFA limit its application of instant filing treatment to documents where such treatment has been specifically requested and where the appropriate transmittal letter and representations have been provided. Although the Division supports the DDRT's goal of completing its reviews of disclosure documents in an expedited fashion and its efforts to encourage qualifying CPOs and CTAs to use the instant filing procedure, the Division believes that DDRT should not grant instant filing treatment to documents for which instant filing relief has not been requested or where the qualifications for instant filing treatment have not been met.
������� NFA procedures set forth specific goals for DDRT staff regarding the amount of time in which review of disclosure documents should be completed. In its Interpretive Notice regarding initial implementation of the Program, NFA stated that disclosure document reviews would generally be completed within 14 days.[39] NFA reiterated this 14-day review policy in the written materials for its first town meeting on disclosure document issues[40] and indicated that a seven-day turnaround is typical. The DDRT Procedures Manual specifies that documents which qualify for instant filing treatment are to be turned around within three days; NFA staff indicated in interviews that, in practice, they review instant filings within three business days.[41] Electronic filings are treated as regular filings with respect to turnaround time. These time periods are consistent with Commission and NFA rules which require that a document be filed with NFA at least 21 days prior to the date of its intended use.[42]
�������
������� Division staff tested the amount of time NFA took to review all types of disclosure documents, whether standard, instant,[43] or electronic,[44] in order to determine whether the turnaround time was acceptable. Staff reviewed the length of time it took for NFA to complete its review of the 98 disclosure documents in the Division's sample.[45] This was accomplished by searching NFA�s database and noting the date that a document was first received, when a comment letter was sent out (where applicable), when the firm responded to the comment letter, and when an acceptance letter was sent.[46] The Division�s testing disclosed that the average time in which DDRT reviewed regular CTA documents was 6.18 days. For regular CPO documents, the average review time was 6.35 days. Instant filings for CTAs were reviewed in an average of 2.64 days and CPO instant filings were reviewed in an average of 2.07 days.[47] For all CTA documents, DDRT�s average review time was 5.41 days, and for all CPO documents, the average review time was 5.31 days. (See Appendix C for further information on the Division�s analysis of DDRT�s turnaround time.)���
The average turnaround time for the tested documents is within an acceptable range. Indeed, it appears that NFA is adhering to its goal of completing its disclosure document reviews in an expeditious manner. While Division staff has concerns about the large number of cases where there was insufficient information to adequately measure the turnaround time, many of these concerns would be addressed if NFA adopted stricter internal controls regarding its data entry and filing procedures. See the �Tracking� section, below.
�
������� The operation of a tracking system is important to the operation of any compliance program. The tracking system allows management to ensure that all documents are processed, and processed in a timely manner. It also allows management to maintain control over work allocation and to monitor staff productivity.
������� Pursuant to DDRT procedures, a disclosure document filing is logged into NFA�s computer system by the DDRT Administrator immediately upon receipt by NFA.[48] The Administrator will assign the filing to a particular reviewer based upon staff workload and availability. The filing may, for continuity purposes, be assigned to a reviewer who reviewed a previous document from the same registrant. The assigned reviewer is responsible for all subsequent communications and data entry with respect to that filing.
������� When a disclosure document is received it is designated as a "NEWDDOC" in the DDRT Database. Documents that respond to comments made by the reviewer are designated as "REVISIONS" to the previously filed document. DDRT�s procedures manual notes that while an initial filing by a firm is not given a specific notation designating it as such, the reviewer should recognize it as an initial filing by noting that there is no file folder for the firm and by noting that there are no prior filings in the computer system. Electronic filings are noted by a checked box labeled �E-filed� in the tracking system.
�
������� Division staff found that many of the procedures described above are often not followed by DDRT staff. For example, in the great majority of documents reviewed by Division staff, the �NEWDDOC� designation was assigned to every incoming document, including revisions to a disclosure document. In addition, the �E-filed� box was checked when a disclosure document was electronically filed, but, in other cases, was checked when comments to a regular filing were sent electronically by DDRT staff.
Division staff tested NFA�s tracking of disclosure document filings by reviewing the sample of disclosure documents to evaluate whether the information recorded in the DDRT Database was consistent with information contained in NFA�s hardcopy files. Division staff found several discrepancies between the dates of correspondence (both to and from NFA) recorded in the DDRT Database and the dates found in NFA�s hardcopy files. In addition, there were many cases in which certain actions were noted in one location but corresponding information was not found in the other location. For example, there were many instances where a document was referred to in the DDRT Database but could not be found in hardcopy files; or, if a document was in the files, where the date stamp on the document differed from the date that was entered into the database. In other cases, there were comment and/or acceptance letters found in the files but not recorded in the database, or vice-versa.
������� Of the 98 documents reviewed by Division staff, 19 contained discrepancies of more than one day between the dates of the disclosure documents and related correspondence recorded in the database and the dates found in the hardcopy files.[49] In six additional cases, staff was unable to determine whether or not discrepancies existed because there was no acceptance letter in either the database or the files.
In 25 of the 98 cases, correspondence was referred to in the database but could not be found in NFA's hardcopy files. In eight other cases, correspondence found in the files was not entered into the database.[50] Therefore, in 38 of the 98 cases (39%) there was a misalignment between the database system and the files.[51]
Division staff believes that many of these errors are the result of the fact that many different NFA staff members are responsible for entering certain data in the system and filing documents and letters in the files. Although the responsibility for logging in disclosure documents has recently been assigned to an administrative assistant, data regarding all subsequent correspondence relating to each document review is still entered in the database or the files by several different staff members. It is our recommendation that NFA devise a system whereby adequate internal controls exist which would reduce the incidence of errors and make it easier to track the �review life� of disclosure documents.
������� It also appears that when DDRT receives an electronic filing, the disclosure document is printed and filed, and the electronic copy is discarded. In light of the increasing use of electronic media in the current business environment, the Division suggests that NFA should consider retaining the documents on disk rather than storing hard copies in the files.[52]
The DDRT checklist requires that the reviewer verify that certain information in the disclosure document is consistent with corresponding information contained in NFA�s databases. The information includes: the name, address and telephone number of the firm; the registration status of the firm; and the registration and disciplinary history of the firm�s principals. This information must be accurately disclosed so that prospective investors may be informed of the fitness of principals of the CPO or CTA and be able to contact the firm should any questions or problems arise. The reviewer is also required to check for the existence of any exemptions from Commission regulations, such as those pursuant to Rule 4.7 or Rule 4.12(b).
DDRT staff indicated in interviews that this verification should be performed for both new and updated documents, and should be performed even if the reviewer does not use a checklist. DDRT staff also indicated that, depending on the nature of the information in the disclosure document, an expanded search of the databases might be conducted.
������� Division staff tested DDRT�s use of the databases by comparing each of the 98 disclosure documents against information contained in MRRS, FACTS and Clearinghouse. Division staff discovered 43 discrepancies between the information in NFA�s databases and the disclosure documents. Of these, 24 (56%) were documented by DDRT staff. Of the 14 cases of discrepancies in documents for which a checklist was prepared, ten (or 71%) were documented by DDRT staff. Of the 29 cases of discrepancies in documents for which a checklist was not prepared, 14 (or 48%) were documented by DDRT staff.
������� As discussed in Recommendation 1, there appears to be a pattern where problems are more likely to be found and documented in cases where the checklist is used. The inverse also appears to be true; the lack of a checklist indicates a greater likelihood that problems are not documented.
������� The work performed by DDRT staff in reviewing disclosure documents and the work performed by audit team members during an audit are complementary and require that the two groups work together in certain areas. When reviewing documents, DDRT staff may note items that, on a paper review, cannot be found to violate any of the compliance rules, but which trigger �red flags� indicating that the items should be reviewed during an audit. Examples include hypothetical performance results, a significant increase or decrease in funds under management, and excessive losses.�� DDRT staff would, by a referral, advise audit staff that they should audit the firm in the near term. In less urgent cases, DDRT staff may assign �audit priority points� to a firm whose document they have reviewed. These audit priority points may cause the firm to be audited earlier than it would otherwise be.[53] Similarly, DDRT staff need to take account of the work of audit staff. For example, during an audit, staff may note problems in a disclosure document which need not be corrected immediately, but which should be corrected when the document is next updated.
Division staff reviewed this coordination in both directions. While it appears that audit staff took note of and acted upon referrals from DDRT staff, it appears that, in a number of cases, DDRT staff failed to comment upon disclosure documents where registrants failed to correct problems noted in recent audits.
The first segment of testing in this area focused on the manner in which audit staff acted upon referrals from the DDRT. During the target period there were four referrals from DDRT staff to audit staff that focused on items which, according to the judgment of DDRT staff, required closer scrutiny of the firms' operations.[54] Division staff agrees that these referrals were all relevant and appropriate.
It appears that the referrals were taken into account by audit staff when choosing firms to audit. Three of the four firms have been or are in the process of being audited by NFA since the referrals were made.[55] Of the three audits, two specifically stated in the audit files that the referral from DDRT was a factor that audit staff used in selecting the firm for an audit. The third did not contain an explicit reference to a referral from DDRT; however, the audit did examine the issues discussed in the referral. Division staff concludes that this segment of the Program is working well.
�
The second segment of testing in this area focused on the use by DDRT staff of the audit priority system, which governs when NFA schedules audits of firms. In general, firms with a greater number of points are audited sooner. DDRT may add priority points based on issues found in disclosure document reviews. One to three priority points may be added for issues such as: exceptionally positive or negative performance; presentation of proprietary or hypothetical performance; unlisted principals; unusual fees; questionable disciplinary history; large increases or decreases in the amount of funds under management; highly leveraged programs; and programs that use an unusual mix of trading instruments. The DDRT Procedures Manual states that, in most cases, no points will be added.
It appears that DDRT staff made only one adjustment to a firm's total number of audit priority points out of the 98 cases in the sample. The one case involved a CTA's use of hypothetical performance data, and three audit priority points were added by the reviewer. The assignment of audit priority points by DDRT staff is a highly discretionary function, and the Division found no abuse of that discretion in the cases in the sample.��
������� The third segment of testing in this area focused on whether DDRT staff properly used the results of recent NFA audits in their review of disclosure documents. Audit team members are required to review a firm�s most recent disclosure document and ensure that the information contained therein comports with other data found at the firm. If the audit team finds discrepancies between the disclosure document and information in the firm�s books and records (or any other areas in which the disclosure document violates either Commission or NFA rules), the firm is informed of the problem in the audit report. In serious cases, the firm will be required to correct the problem immediately. In less serious cases, the firm may be instructed to correct the problem when it files its next disclosure document. DDRT procedures require staff to consult the audit files for any audit conducted during the previous year to ensure that all issues raised in the audit report have been corrected in the disclosure document under review.
For the 98 firms in the Division's sample, there were 24 audits conducted during the target period and the prior year.[56] Of these 24 audits, ten noted deficiencies in the disclosure document which were required to be corrected before a subsequent version could be distributed. In six of these ten cases, the deficiencies were corrected when the firm filed its subsequent disclosure document. Thus, there were four disclosure documents in the sample in which the firm had failed to correct deficiencies noted in the recent audit.
In one of these four cases, the DDRT reviewer noted the continued deficiency when the subsequent disclosure document was sent to NFA, and required the firm to correct it. In the other three cases where the disclosure document under review did not correct deficiencies noted in the recent audit, DDRT staff failed to raise the issue during their review of that filing.
While the Division cannot draw a firm conclusion from a sample of only four cases, these results are not encouraging. The Division recommends that DDRT procedures for consulting audit files be followed more rigorously.
In the course of reviewing disclosure documents, the reviewer may discover that a registrant has developed a new and innovative way of packaging a trading methodology, combining commodity and non-commodity investments, or marketing the program being offered. There is no way to predict when, or from whom, such items may appear.
In accepting the delegation of the review program, NFA, indicated to the Commission that it would keep the Commission apprised of emerging disclosure issues and to consult the Commission in developing policies to handle such issues.[57] DDRT staff regularly contact Division staff, on a case by case basis, to discuss questionable items that may need greater clarification. In at least one case in the sample, NFA staff worked closely with Division staff in clarifying a new and innovative pool structure and ensuring that adequate disclosures were made.
In the course of this review, however, Division staff noted another document with a �new and innovative� issue. This case involved a disclosure document for a "family" of six pools, each of which had a different leverage and fee structure. The document raised issues regarding potential commingling of the different pools and invasion of one pool to cover losses in another of the family, due to the unlimited liability of participants in the pools. It appears that DDRT staff did not consult with Division staff regarding this document, and it was accepted for use. On the basis of its review of this document, Division staff believe that the document should have provided expanded disclosure of the risk factors attributable to this structure and the extent of participants� liability. In addition, there were problems with the presentation of the pools� break-even analysis that DDRT staff failed to note.
As noted above, some of the compliance problems identified by Division staff were not identified by NFA staff due to differences in views concerning the requirements of the Commission�s regulations. To ensure that both mature and emerging disclosure and regulatory issues are fully considered, the Division recommends that Division staff and NFA staff work together to schedule periodic meetings to discuss such issues. The Division acknowledges that issues posed by individual disclosure documents must be addressed by NFA as they arise, and encourages DDRT to continue to consult with Division staff on an as-needed basis.
�
Division staff reviewed the experience and training of DDRT staff to ensure that appropriate resources were being applied to the disclosure document program. The Division found that the NFA staff responsible for the Program were adequately trained and had sufficient experience to carry out the Program.
������� The DDRT is composed of eight staff members, of whom three are part-time employees,[58] with average experience of nearly ten years with NFA. All DDRT staff had at least two years of experience as auditors in NFA's Compliance Department prior to joining the DDRT. The range of NFA experience of the staff that currently comprise the DDRT is four to 15 years. Six staff members have a Bachelors of Science in Finance, one staff member has a Bachelors of Science in Accounting and one staff member has a Masters in Business Administration. All eight members of the DDRT staff have been members of the DDRT since its inception. Two additional staff members have left the DDRT but have not been replaced.
������� Division staff interviewed all eight staff members regarding their training. All members of the DDRT expressed the belief that their training prior to joining the DDRT provided them with a sufficient background to join the team. The staff indicated that they had received substantial in-house training on a range of industry topics before becoming members of the DDRT. In addition, the staff indicated that they had attended several training sessions with CFTC staff during the initial implementation of the Program. Although most of the in-house training deals with broader compliance topics, staff indicated that they have had a limited amount of specialized training, including sessions on the �fully-funded subset� and promotional materials. Staff have received both in-house and outside training in areas such as options and risk management.
������� DDRT staff members indicated that they all had participated in at least one of the town meetings held by the NFA to allow the industry to become familiar with the NFA staff who would be carrying out the review program and to acquaint the industry with NFA's approach to the review program. Some of the staff indicated that they had also prepared materials for presentation at the town meetings. The staff indicated that they all had attended at least one industry conference since the program's inception, which provided information that has been particularly useful, given the specialized nature of this program. Finally, DDRT staff indicated that the weekly meetings held within the DDRT to discuss issues that had arisen during the prior week were particularly valuable in enabling staff to exchange information regarding the handling of these issues.
������� The Division concluded that NFA's design and execution of its Program complies with applicable regulatory standards and is generally effective in achieving its intended purposes. The Division's recommendations for enhancement of the Program are based primarily on NFA�s demonstrated ability to conduct timely and effective disclosure document reviews, when its procedures, particularly use of the checklist, were followed. The Division believes that the suggested enhancements of NFA�s procedures for disclosure document review, documentation of communications with members, instant filing, and data entry and maintenance will enable NFA to improve the efficiency and effectiveness of its Program.���
�
Appendix B: CPO Disclosure Document Checklist
�
Appendix D: Statistical Analysis of Differences
�
[1] The Division�s most recent prior reports on NFA�s programs were in June 1999, September 1998 and July 1997. The 1999 report covered NFA's program for the supervision of telemarketing activities; the 1998 report covered NFA's compliance program for futures commission merchants ("FCMs�) and introducing brokers ("IBs"); and the 1997 report covered NFA�s compliance program for commodity pool operators (�CPOs�) and commodity trading advisors (�CTAs�).
[2] Commission Rules 4.26(d) and 4.36(d).
[3] NFA Compliance Rule 2-13 requires that NFA members file with NFA a copy of any document required to be filed with the Commission pursuant to Part 4 of the regulations.
[4] 44 Fed. Reg. 1918 at 1920 (January 8, 1979).
[5] Pursuant to Rule 4.24(d)(3)(i), privately offered commodity pools are those offered pursuant to section 4(2) of the Securities Act of 1933, as amended (15 U.S.C. 77d(2)), or pursuant to Regulation D thereunder (17 CFR 230.501 et seq.). A publicly offered commodity pool refers to a distribution of units, some or all of which are registered under the Securities Act of 1933. The Commission and the Securities and Exchange Commission ("SEC") have shared jurisdiction over the regulation of commodity pools, as the sale of an interest in a commodity pool is deemed to be a "security." The Commission determined to retain responsibility for review of public pools after consultation between Division and SEC staff in connection with NFA�s proposal to assume the disclosure document review function.
[6] 62 Fed. Reg. 52088 (October 6, 1997). The delegation order also authorized NFA to process certain exemption notices filed pursuant to Part 4 of the Commission's regulations. These functions were not included in the Division's review.
[7] This means that the likelihood that this difference is due to sampling error, rather than a true difference between the two groups of documents, is less than one percent. See, e.g., G. Snedecor & W. Cochran, Statistical Methods 124-25 (7th ed. 1980). Further detail concerning the statistical analysis is provided in Appendix D to this report.
[8] CFTC Advisory 95-44 (Comm. Fut. L. Rep. (CCH) �26,385 (April 20, 1995).)
[9] In one additional CTA document, NFA documented that a compliance problem was found, and that the problem was fixed.
[10] Public Law 93-463, 88 Stat. 1389.
[11] See supra, note 2.
[12] Pursuant to Rule 4.21(a), a CPO may not solicit, accept or receive funds, securities or other property from a prospective participant in a pool that it operates or intends to operate unless, on or before the date it engages in that activity, the CPO delivers or causes to be delivered to the prospective participant a Disclosure Document for the pool containing the information set forth in Rule 4.24.
[13] Pursuant to Rule 4.31(a), a CTA may not solicit or enter into an agreement with a prospective client to direct or to guide the client's commodity interest account or trading unless, at or before the time it engages in the solicitation or enters into the agreement (whichever is earlier), the CTA delivers or causes to be delivered to the prospective client a disclosure document for the trading program containing the information set forth in Rule 4.34.
[14]� Commission Rules 4.26(d)(1); 4.36(d)(1). Rule 4.8 provides an exemption from the 21-day prefiling requirement of Rule 4.26(d)(1) to CPOs with respect to pools offered or sold solely to �accredited investors� in an offering exempt from registration under the Securities Act of 1933, as well as to CPOs of 4.12(b) pools in an exempt offering under the Securities Act of 1933. Commission Rule 4.12(b) provides relief from certain Part 4 requirements with respect to pools that will, in part, not enter into commodity futures and commodity options contracts for which the aggregate initial margin and premiums exceed ten percent of the fair market value of the pool�s assets.
[15] Rules 4.26(a)(2); 4.36(b).
[16]� These practices are in accordance with the Commission�s instructions. See 46 Fed. Reg. 26004, 26010 (May 8, 1981).
[17]� NFA Compliance Rule 2-13 was adopted effective September 29, 1982.
[18] NFA continues to review CPO and CTA disclosure documents during its field audits.
[19] NFA�s desk reviews covered areas such as performance presentation, registration and disciplinary information, unusual investment strategies, fees and expenses, and conflicts of interest. The firm's compliance files were also reviewed for Division exemption and no-action letters. Where NFA's desk review noted problems, audit priority points could be assigned (see below at p. 21).
[20] Division of Trading and Markets Report, �Review of the CPO and CTA Compliance Program of National Futures Association,� July 29, 1997.
[21] Letter from Brooksley Born, Chairperson of the Commission, to Robert K. Wilmouth, President of NFA, dated May 28, 1997.
[23] See supra, note 6.
[24] Information is entered into MRRS by NFA, firms who participate in direct entry, exchanges and the CFTC. Almost all of the information included on the CFTC Form 7-R and CFTC Form 8-R can be found in MRRS.
[25] The Clearinghouse includes information on: all disciplinary actions taken by NFA since its inception; all administrative and injunctive actions taken by the Commission since 1975; and all formal disciplinary actions issued by the U.S. futures exchanges since 1990. It does not include civil actions between parties, criminal proceedings, actions brought by state agencies, or actions brought by securities self-regulatory organizations. Information available through the Clearinghouse is provided to NFA by contributing organizations, which include the Commission and U.S. futures exchanges. Disciplinary information is accessible to the public through a toll-free phone number. In February 1999, NFA began providing disciplinary information on its web site through the Background Affiliation Status Information Center ("BASIC").
[26] The sample size was selected using a statistical analysis, balancing the priorities of obtaining a sample large enough to draw significant conclusions but not so large as to be impractical to analyze. The Division�s sample was selected from a list of all disclosure documents filed with DDRT during the target period. At the time of the Division�s on-site review, NFA was unable to find one CTA document and six CPO documents. The Division located copies of three of the missing documents in its own files. The remaining documents were replaced by random selection from the list. DDRT staff subsequently informed the Division that data entry errors were responsible for their inability to locate five of the seven documents at the time of our on-site review. The sixth document was lost, and later replaced. DDRT could not determine why the seventh document was not found during the on-site review, but they confirmed that the document was indeed in their files. Cf. Recommendation 3.
�
[27] Division staff also checked MRRS and the Clearinghouse for registration and disciplinary information regarding the firms and their principals, and later cross-checked the audit information provided by NFA staff. (See Sections 6 and 7, below.)
[28] The DDRT Procedures Manual explicitly states: "If [comments are] communicated orally, documentation of such oral communication must be made in the correspondence section of the checklist." While it is possible that DDRT communicated comments concerning some deficiencies orally, it is impossible for the Division to review any undocumented comments. Accordingly, the Division has based its evaluation of DDRT's reviews solely on documented comments.
[29] It is possible that a combination of several less significant issues in a document could be sufficient to render a document incapable of use. Division staff did not, however, find any cases in which NFA missed a number of less significant issues which, taken together, would have been enough to preclude use of the document for solicitation.
[30] DDRT staff also noted that checklist items specifying data that the reviewer should enter in the FACTS system are not currently being used since NFA is in the process of converting FACTS to an Access database. Division staff were informed that this information is being added to FACTS from the annual membership questionnaire.
[31] The differences for both types of problems in CTA documents were statistically significant at a confidence level well above 99%.
[32] This difference was statistically significant at a confidence level of 90%.
[33] This comparison, while not statistically significant, did favor the non-checklist group.
[34] The Division examined the pattern of missed problems in the 44 documents where NFA�s notes identified limited areas of change, as well as in one other document where the transmittal letter identified limited changes. The Division found that a high majority of the missed problems (39 of 54, or 72%) were in the areas identified as changed. Removing the remaining 20 missed problems from the analysis raises the percentage of problems caught in documents without checklists from 54% to 65%, but the difference from the 87% found in documents with checklists is still significant at a confidence level well above 99%.
[35] Given the significant number of missed problems (20) in areas not identified as changed, NFA may wish to consider adopting a policy of subjecting each disclosure document to a full scope review periodically (perhaps every two years).
[36] See supra, note 8.
[37] Instant filing relief is also available for disclosure documents of pools that will be operated pursuant to an exemption under �4.12(b) of the Commission�s regulations and which invest in futures contracts and options thereon exclusively through investments in other �4.12(b) pools. This relief is based on the �limited and indirect nature of the investment� in futures contracts and commodity options. None of the CPO documents in the Division�s sample were filed pursuant to this provision of Advisory 95-44.
[38] NFA�s notes for these reviews indicated that the document was treated as an instant filing. Division staff examined the transmittal letters contained in NFA�s files and ascertained that neither the requisite request nor the representations were provided in any of the 13 cases.
[39] NFA Notice I-97-22 (October 20, 1997).
[40] NFA held �town meetings� on disclosure document issues in Chicago on December 10, 1998 and in New York on April 29, 1999. Additional town meetings are scheduled for October 6, 1999 in Los Angeles and October 7, 1999 in San Francisco.
[41] The guideline for instant filings is similar to the time frame set forth in the Commission�s instant filing advisory, which stated that instant filings would be reviewed �generally within two calendar days of the filing date.� See supra, note 8.
[42] Commission Rules 4.26 and 4.36; NFA Compliance Rule 2-13.
[43] For purposes of determining turnaround time, the Division included all documents in the sample that DDRT treated as instant filings, regardless of whether instant filing treatment was requested or warranted. Thus, 11 of the 49 CTA documents in the sample, and 16 of the 49 CPO documents in the sample, were analyzed as instant filings.
[44]� While the Division initially distinguished standard and electronic filings in its review of DDRT�s turnaround time, there were no material differences in the time in took DDRT to review documents filed electronically as compared to other filings. Electronic filings are therefore not noted separately.
[45] In determining turnaround time, the Division measured the total �review life� of each disclosure document in the sample. The �review life� begins the day that NFA receives the disclosure document and ends the day the disclosure document is accepted. Only the amount of time that the document is being worked on by NFA staff is counted; the time spent by the firm making revisions and sending the revisions to NFA is not included. Each step in a document�s �review life� is added together to arrive at a turnaround time.
[46] Note that there were four CPO documents and two CTA documents where it was impossible for the Division to determine the turnaround time because there was neither an acceptance letter nor an indication, in either the database or the files, that the document had been abandoned by the submitter. These cases were noted by the Division but not included in the composite average numbers.
[47] This figure is the median of all instant filings for CPOs. Division staff felt that this figure was more representative of the turnaround time since the sample included an �outlier� of 27 days. Turnaround time for the other 15 instant filings was 0-6 days.
[48] During the target period, the DDRT Administrator function rotated among the DDRT staff, with a different individual responsible for logging in documents each week. The testing and conclusions discussed here are based on documents processed during this period. Beginning in June 1999, the DDRT Administrator function has been assigned to a single administrative assistant .
[49] In five additional cases, there was a discrepancy of one day.
[50] In one case, both types of error were present.
[51] This includes the six cases in which there was no acceptance letter in either the database or the files.
[52] One of NFA's regulatory initiatives for the upcoming year is adapting rules to a changing business environment. With respect to this initiative, NFA has stated: �Existing NFA and CFTC rules were, for the most part, written for a 'paper world' which becomes less relevant every day. As electronic exchanges and electronic order routing become more prevalent, there is definitely a pressing need for a comprehensive update of the existing rules to reflect the changing business environment.� NFA News, Facts, Actions, Vol. 17, No. 3 (May - June 1999).����
[53] In rare instances, DDRT staff may request records from the firm and perform a limited review of the firm�s records. DDRT staff estimated that the team has done so approximately twice since the inception of the Program.
[54] Two of the referrals focused on unusually high hypothetical performance results in the firms' initial disclosure documents (in both cases, the rates of return were well over 100 percent per year). A third referral was made where a CTA had large percentage losses for several successive months but continued to attract new capital. The fourth was based on a CTA who withdrew his NFA membership for a six-month period while acting as an information-only CTA, then reinstated it afterwards. NFA Bylaw 1101 prohibits an NFA member from �doing business with� any non-member required to be registered with the Commission that is acting with respect to an account, order or transaction for a customer, a commodity pool or participant therein, a client of a CTA, or any other person.
[55] In one case the DDRT referral was made in August 1998 and the audit took place April 1999, in the second case the referral was made in July 1998 and the audit took place in June 1999. In the third case Division staff was unable to determine the length of time between the referral and the audit because the referral memo was not dated.
[56] This does not include audits conducted after the document in the sample was reviewed.
[57] See Letter dated June 20, 1997, from Robert Wilmouth to Brooksley Born at page 1 ("Under our proposal we would seek input from the Commission staff on emerging disclosure issues through both formal and informal channels. . . . We would suggest periodic meeting between NFA and CFTC staff to discuss the issues that have been identified by either." )
[58] Two of the DDRT staff members work three days a week, and one works four days a week.
[59] The range of values for the CTA disclosure documents was as follows: for standard filings, the range was 0-24 days, and the range for instant filings was 1-6 days.
[60] The range of values for the CPO disclosure documents was as follows: for standard filings, the range was 0-43 days, and the range for instant filings was 0-27 days. One entry failed to document the amount of time the firm took to respond to comments from NFA. In this case, Division staff calculated the total review life to include the total time between the comment letter and the acceptance letter, which was seven days.
[61] This figure is the median of all instant filings for CPOs. Division staff felt that this figure was more representative of the turnaround time since the sample included an outlier of 27 days. Turnaround time for the other 17 instant filings was 0-6 days.