Appendix D: Statistical Analysis of Differences
�
|
All Documents -- All Issues |
||
|
NFA with Checklist |
NFA Without Checklist |
|
| Number found |
61 |
62 |
| Total issues |
70 |
115 |
| Percentage found |
87.14% |
53.91% |
| Difference in percentage |
33.23% |
|
| Confidence level |
99.9997% |
|
�
|
CTA Documents -- All Issues |
||
|
NFA with Checklist |
NFA Without Checklist |
|
| Number found |
43 |
20 |
| Total issues |
47 |
58 |
| Percentage found |
91.49% |
34.48% |
| Difference in percentage |
57.01% |
|
| Confidence level |
99.9999997% |
|
�
|
� � |
CTA Documents -- More significant Issues |
|
|
NFA With Checklist |
NFA Without Checklist |
|
| Number found |
16 |
8 |
| Total issues |
18 |
22 |
| Percentage found |
88.89% |
36.36% |
| Difference in percentage |
52.53% |
|
| Confidence level |
99.93% |
|
�
|
CTA Documents � Less significant Issues |
||
|
NFA With Checklist |
NFA Without Checklist |
|
| Number found |
27 |
12 |
| Total issues |
29 |
36 |
| Percentage found |
93.10% |
33.33% |
| Difference in percentage |
59.77% |
|
| Confidence level |
99.9999% |
|
�
|
CPO Documents -- All Issues |
||
|
NFA with Checklist |
NFA Without Checklist |
|
| Number found |
18 |
42 |
| Total issues |
23 |
57 |
| Percentage found |
78.26% |
73.68% |
| Difference in percentage |
4.58% |
|
| Confidence level |
33% |
|
�
|
CPO Documents -- More significant Issues |
||
|
NFA With Checklist |
NFA Without Checklist |
|
| Number found |
4 |
6 |
| Total issues |
4 |
11 |
| Percentage found |
100.00% |
54.55% |
| Difference in percentage |
45.45% |
|
| Confidence level |
90% |
|
�
|
CPO Documents -- Less significant Issues |
||
|
NFA With Checklist |
NFA Without Checklist |
|
| Number found |
14 |
36 |
| Total issues |
19 |
46 |
| Percentage found |
73.68% |
78.26% |
| Difference in percentage |
4.58% |
|
| Confidence level |
31% |
|
The test for statistical significance between two rates (in this case, catching A out of a total of B problems versus catching C out of a total of D problems) is performed as follows:
Let
P1 = A/B
P2 = C/D
P* = (A+C)/(B+D)
Then the test statistic (Z) will be equal to Numerator/Denominator, where
Numerator = the absolute value of (P1 � P2) and
Denominator = the square root of [(P*) * (1-P*) * ((1/B) + (1/D))]
And the confidence interval will equal the area under the standard normal curve at Z, using a two-tailed test.