Medicare and Medicaid Audit Sampling...

Medicare and Medicaid Audit Sampling Strategies Developing Sampling Plans and Challenging Flawed CMS Audit Samples

Today’s faculty features:

1pm Eastern | 12pm Central | 11am Mountain | 10am Pacific

The audio portion of the conference may be accessed via the telephone or

by using your computer's speakers. Please refer to the instructions emailed

to registrants for additional information. If you have any questions, please

contact Customer Service at 1-800-926-7926 ext. 10.

TUESDAY, AUGUST 14, 2012

Presenting a live 90-minute webinar with interactive Q&A

Anna M. Grizzle, Member, Bass Berry & Sims, Nashville, Tenn.

Patricia L. Maykuth, Ph.D, President, Research Design Associates, Decatur, Ga.

PLEASE NOTE: This PDF is 697 pages long.

For printing purposes, use the Bookmarks Panel to navigate to the

pages you want to print.

Research Design Associates (404) 373 4637 721 E Ponce de Leon www.researchdesignassociates.com Decatur, GA 30030

Notes to Accompany Stafford Publishing Co webinar: Medicare and Medicaid Audit Sampling Strategies

Tuesday, August 14, 2012 Definitions Slide 11 Numbers can readily be manipulated and outcomes understood through the use of simple math: addition, subtraction, multiplication, multiplication and division e.g., %s, differences, sums and averages. Statistics is branch of applied math concerned with the collection and interpretation of quantitative data and the use of probability theory to estimate universe parameters e.g. correlations, t‐tests and point estimates

Inferential Statistics is branch of applied statistics drawing conclusions about a population from a random sample drawn from it. These mathematical analyses move beyond mere description of research data to make inferences about the larger population from which the sample was drawn. This branch of applied statistics is used to drawing conclusions about a population from a random sample drawn from it. Allow projection of overpayment.

Sample necessary for extrapolation: A sample chosen from independent observations that are randomly selected from a normally distributed universe to yield a representative set of observations. All have to be tested mathematically. The sample must be large enough to reflect the statistical characteristics of the universe and for CMS work be associated with chosen precision and confidence. The result is that each selected observation is equally likely to be selected as any other observation yielding an unbiased set of observations for drawing inferences about unknown characteristics of the universe. A simple random sample is selected so that all samples of the same size have an equal chance of being selected from the population. Sometimes within a universe there are groupings of similar data that can be divided into sub categories. Stratified sampling involves selecting independent samples from a number of subpopulations, group or strata within the population. Great gains in efficiency are sometimes possible from judicious stratification. This results in reduced error rates and small sample sizes which can efficiently and accurately represent the universe groupings. A probability sampling scheme is one in which every unit in the population has a chance (greater than zero) of being selected in the sample, and this probability can be accurately determined. The combination of these traits makes it possible to produce unbiased estimates of population totals, by weighting sampled units according to their probability of selection. When the probability of a claim’s occurrence it the universe is known, the claim is weighted in accordance to its probability of selection. Therefore, what makes it a probability sample is the fact that each claim’s probability of selection is known. If every claim has the same probability of selection from the universe it is known as an 'equal probability of selection' (EPS) design. Simple random samples are such designs. Such designs are also referred to as 'self‐weighting' because all sampled units are given the same weight. Probability sampling includes: Simple Random Sampling, Systematic Sampling, Stratified Sampling, Probability Proportional to Size Sampling, and Cluster or Multistage Sampling.


Some Common CMS statistical terms Slides 18 ‐21, 35 Universe: all billing in target timeframe from CMS payment data Sampling unit – the unit of measurement for the study: claim, beneficiary, specific payment codes Claim: either aggregated Claim, all lines of data billed for one patient on a given day or claim, a single line of data. The auditor must stick with the chosen unit & not switch back and forth Frame: selected parts of the universe used in entirety. (All Claims for the 5 CPT codes chosen for audit) Parametric tests make specific assumptions about the population parameters that characterize the underlying distributions for that test Non parametric tests make few or no assumptions about the underlying distribution of the and parameters of the population Sample: randomly chosen sub set of claims which get audited Overpayment: calculation and extrapolation Point estimate: average (mean) overpay amount RAT‐STATS – DHHS provided statistical package to calculate sample size, random number table, and appraisals (point estimates). Error terms

Variance ‐ distance between each set of data points and their mean Standard deviation – square root of the variance Precision ‐ accuracy of measurement, closeness to true universe value Precision % ‐ one measure of CoV Coefficient of Variation ‐ normalized measure of dispersion of the probability distribution or

relative standard error Error rate –proportion of claims in error Confidence Interval ‐ likelihood that chose the right claims & projection is accurate

Precision and lower bounds Sampling error –discrepancy between the value of a statistic and the parameter it estimates

MPIM ‐ Medicare Program Integrity Manual Mean (average) the arithmetic sum of all scores divided by the number of cases Median – the middle most real score in the data set Mode the score that occurs most frequently in the data set (does not have to be unique – sometimes more than 1 value is equally likely) Measures of variability Normal Distribution bell‐shaped distribution necessary for use of parametric statistics Null Hypothesis a statistical hypothesis that is tested then accepted or rejected; specifically : the hypothesis that an observed overpayment amount is due to chance alone and not due to a systematic or biasing cause Parameter refers to a DISTRIBUTIONAL characteristic of a population

• Parametric Statistics are probability estimates based on the parameters of a normal distribution. Parametric tests make specific assumptions about the population parameters that characterize the underlying distributions for that test

• Non Parametric Statistics tests make few or no assumptions about the underlying distribution of the and parameters of the population

Point Estimate uses sample data to calculate a single point which serves as the best estimate of a universe parameter Posterior Distributions‐ In Bayesian statistics, the posterior probability of a random event for an unknown quantity conditional on the evidence obtained from an experiment. That is the overpayment amount from an audited sample of Medicare claims. Statistic refers to a characteristic of a sample, descriptive or inferential Representative sample must cover all salient features of the universe without overlapping segments to be a true picture of the universe from which it was selected and over which projections will be made.


Some MPIM Language and Its Statistical Meaning Slide 13

If a particular probability sample design is properly executed, i.e., defining the universe, the frame, the sampling units, using proper randomization, accurately measuring the variables of interest, and using the correct formulas for estimation, then assertions that the sample and its resulting estimates are “not statistically valid” cannot legitimately be made. In other words, a probability sample and its results are always “valid.” MPIM § 8.4.2

Slide 14 The “always valid” concept refers to the idea that internal operation of a statistical process which will (with respect to its mathematical assumptions) yield internally consistent results when executed. It is the internal consistency which is “valid,” rather than whether the use of the results will be appropriate. “Valid,” as used in the MPIM does not mean that a given internally consistent statistical result can be used for all purposes. The perception that the process, procedure and results of any audit cannot be challenged is a misinterpretation of statistical science and the MPIM. Rejecting the findings (the hypothesis) is a legitimate scientific result of any study. The concept of statistically “valid” includes the understanding that there is an expectation of error. For example, one may have a sample used for predicting heights of mountains which produces mathematically internally consistent results and hence is statistically “always valid,” but those internally consistent results are not, in plain English, valid for predicting distances to planets. It is simply not valid for the measurement of the distance to a planet and the results do not fit the “always valid” criteria. Similarly, internally consistent parametric methodologies are not, in plain English, valid for making predictions from “non normally distributed” data. The assumptions and steps required for accurate mathematical calculation of error, confidence and precision all have to be adhered to. One “valid result” can be to reject the hypothesis. Another can be that the results obtained fall outside boundaries of the probability statement and thus should not be used. Further a “valid finding” could be that the (a) error rates from a “valid sample” are too high or (b) the samples were unrepresentative. A “valid result” can be to reject the finding, in other words, the conclusion “the findings should be rejected” is a valid result. +++++++++ An appeal challenging the validity of the sampling methodology must be predicated on the actual statistical validity of the sample as drawn and conducted. Failure by the PSC or ZPIC BI units or the contractor MR units to follow one or more requirements may result in review by CMS of their performance, but should not be construed as necessarily affecting the validity of the statistical sampling and/or the projection of the overpayment. MPIM 8.4.1.1 The resulting lack of a “probability sample” invalided the auditor’s prediction of the amount of alleged overpayment. Accordingly, the auditor’s predicted (1) overpayment and (2) the “lower bound” of the “confidence interval” of such overpayment were scientifically, mathematically, statistically and methodologically invalid. It is too excessive a departure from even a lenient interpretation of the MPIM to allow an auditor to employ parametric statistics with not normally distributed data. To do so meant that the auditor disregarded the key mathematical assumption of parametric statistical methodology, namely, the assumption that only normally distributed data will be used to calculate a parametric statistic. Further sample sizes must be determined based on knowledge of prior error, chosen precision and confidence levels. These are key mathematical assumptions. The integrity of the Medicare claims appeals process is predicated on being able to hold auditors accountable for fundamental, non‐trivial errors.


MPIM (continued).

even a “properly executed” “probability sample” has an expectation of error, being “properly executed”

does not mean that the sample was useful to generate an acceptable degree of predictive accuracy and

reliability. For example, at the 90% confidence level there is the expectation that 1 time in 10 or (10

times in 100) the prediction will be wrong. So even a properly executed “probability sample” could be

wrong. Part of “proper execution” by a statistical researcher is knowing when to reject the hypothesis

(reject the extrapolation). Further, the MPIM does not cover the reality that one “legitimate” result can

be to reject the hypothesis that the estimate is true. Rejecting the extrapolation is a “legitimate” result

when required accuracy is not obtained, bias is discovered or other problems identified.

Boiled down to its essence, this MPIM §8.4.2 states a truism, namely, if a “probability sample” is “validly” drawn, then it cannot be disputed that it is “validly” drawn. Similarly, if one uses a “valid” “probability sample” “properly,” “accurately” and “correctly,” then it cannot be disputed the results of using the sample will be proper, accurate and correct. What appears to a non‐statistician at first to be jarringly illogical is the last sentence of MPIM §8.4.2 which says, in pertinent part, “… a probability sample and its results are always ‘valid.’” The use of “valid” here appears illogical to the non‐statistician because here “valid” is used with its statistical meaning, rather than with its lay English meaning. In plain English, “valid” connotes truth or an adequate multi‐dimensional sufficiency for all pertinent purposes. In English, one might think that you could, as a practical matter, use a “valid” sample for all purposes. However, in the statistical use (including as used in the MPIM), “valid” does not have this meaning. The statistical used in MPIM §8.4.2, “valid” is a one dimensional logical statistical construct where “valid” means that the sample to which it is attached is capable of determining mathematically whether to accept or reject the “null hypothesis.” +++++++ The impression given in the last sentence of MPIM §8.4.2 about a probability sample and its results being “always valid” is that the overpayment number cannot be challenged. This impression is wrong. A “valid” results may be “wrong,” insufficiently “precise,” or not useable to solve the problem at hand. The “always valid” refers to the idea that internal operation of a statistical process which, when executed, will (with respect to its mathematical assumptions) yield internally consistent results. It is the internal consistency which is “valid,” rather than whether the use of the results will be appropriate. “Valid,” as used in the MPIM, does not mean that a given internally consistent statistical result can be used for all purposes. The interpretation that the process, procedure and results of any audit cannot be challenged is a misinterpretation of statistical science and the MPIM. Rejecting the findings (the hypothesis) is a legitimate result scientific result of any study. The concept of statistically “valid” includes the understanding that there is an expectation of error. One only has to look at statistically “valid” weather forecasting statistical results to understand that a “valid” 10% chance of rain will be wrong sometimes. In another example, one may have a methodology used for predicting heights of mountains which produces mathematically internally consistent results and hence the sample is statistically “always valid”. But those internally consistent results are not, in lay English, valid for predicting distances from earth to the planets.


Mock up data for Webinar RAT‐STATS Illustrations Sample Size Determination

Statistics

frame

Valid 70 N

Missing 0

Mean 49.99

Std. Error of Mean 1.380

Median 50.00

Mode 51

Std. Deviation 11.544

Variance 133.261

Skewness .156

Std. Error of Skewness .287

Kurtosis .856

Std. Error of Kurtosis .566

Range 60

Minimum 20

Maximum 80

Sum 3499


Sample size based on chosen Confidence and precision

Sample RAT‐STATS Sample Size Determination Output

I. RAT‐STATS determined sample size for the parameters listed

Confidence Level 80% 90% 95% 99% 1% 75 77 78 79 2% 63 68 71 75 Precision 5% 29 * 39 46 56 Level 10% 10 * 15 * 20 * 29 * 15% 5 * 8 * 10 * 16 *

NOTE (*): One or more sample sizes were under 30. The generated sample sizes were the result of mathematical formulas and did not incorporate management decisions concerning the purpose of the sample or current organizational sampling policies. You may need to increase the sample sizes in order to be in compliance with organizational objectives. NOTE (#): The formulas calculated a sample size greater than the universe size. The program reduced the calculated sample size to the universe size. The additional sampling units were then distributed among the remaining strata based on optimal allocation formulas.


Sample Slide 38 ‐39

Statistics

sample

Valid 15N

Mean 430.93


Median 112.00

Mode 120


Variance 276170.210

Skewness .816


Kurtosis -1.441

Std. Error of Kurtosis 1.121

Range 1290

Minimum 10

Maximum 1300

Sum 6464

Sample Amount Paid

Overpay $

10 10

30 20

100 100

110 110

120 120

1000 1000

1100 1100

1200 1200

1300 1300

120 120

1111 0

25 0

112 0

15 0

111 0


Posterior distribution – Overpayment Slide 40‐41, 44

Statistics

overpay

Valid 15N

Mean 339.3333


Median 100.0000

Mode .00


Variance 261763.810

Skewness 1.197


Kurtosis -.524

Std. Error of Kurtosis 1.121

Range 1300.00

Minimum .00

Maximum 1300.00

Sum 5090.00



RAT‐STATS Point Estimate Output Slide 42 ‐45 This example of the point estimate program output for a stratified random sample using proportional allocation. The point estimate for the overpayment estimate is given for the frame along with the Lower Level at 90% for a 2‐tailed statistical test. Highlighted are the numbers generally found in overpayment demand letters.

Windows RAT-STATS

Statistical Software VARIABLE UNRESTRICTED APPRAISAL Time: 12:54

AUDIT/REVIEW: Webinar SAMPLE SIZE VALUE OF SAMPLE NONZERO ITEMS 15 5,090.00 10 --------------------- D I F F E R E N C E ---------------------- MEAN / UNIVERSE 339.33 80 STANDARD DEVIATION 511.63 SKEWNESS 1.07 KURTOSIS 2.26 STANDARD ERROR (MEAN) 119.08 STANDARD ERROR (TOTAL) 9,526 POINT ESTIMATE 27,147 CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT 14,334 UPPER LIMIT 39,959 PRECISION AMOUNT 12,813 PRECISION PERCENT 47.20% T-VALUE USED 1.345030374455 90% CONFIDENCE LEVEL LOWER LIMIT 10,368 UPPER LIMIT 43,925 PRECISION AMOUNT 16,778 PRECISION PERCENT 61.81% Unacceptable precision T-VALUE USED 1.761310135775 95% CONFIDENCE LEVEL LOWER LIMIT 6,715 UPPER LIMIT 47,578 PRECISION AMOUNT 20,431 PRECISION PERCENT 75.26% T-VALUE USED 2.144786687918

The hypothesis should be rejected and extrapolation thrown out. The hypothesis was that the point estimate was an accurate estimate of the true mean of the universe. The precision asserted to be 10% was ACTUALLY 61.81%. The point estimate and the confidence limits were not.


Links and references

Ethical Guidelines for Statistical Practice, American Statistical Association http://www.amstat.org/about/ethicalguidelines.cfm Worth a look and consideration when criticizing obviously problematic audit execution and conclusions. Add RAT‐STATS program & manuals (free resources) To Download CMS RAT‐STATS program go to http://oig.hhs.gov/compliance/rat‐stats/index.asp or Download RAT‐STATS for Windows (ZIP)

Extract zip files to a folder on your hard drive. Run SetupRS2007v2.exe and follow the installation instructions.

RAT‐STATS is a free statistical software package that providers can download to assist in a claims review. The package, created by OIG in the late 1970s, is also the primary statistical tool for OIG's Office of Audit Services. Among other tasks, the software assists the user in selecting random samples and estimating improper payments. We have attempted to make RAT‐STATS as user‐friendly as possible, keeping in mind the program uses technical statistical terms. OIG does not provide technical support for RAT‐STATS. Although OIG does not require the use of RAT‐STATS, many providers download the software in their efforts to fulfill the claims review requirements for corporate integrity agreements or provider self‐disclosure protocol. Two helpful references RAT‐STATS User Guide http://oig.hhs.gov/organization/oas/ratstats/UserGuide2007.pdf RAT‐STATS Companion Manual http://oig.hhs.gov/organization/oas/ratstats/CompManual2007.pdf


Alphabet CMS contractors in the current audit landscape Recovery Audit Contractors (RACs) Medicare Administrative Contactors (MACs) Medicaid Integrity Contractors (MICs) Program Safeguard Contractors (PSCs) & Zone Program Integrity Contractors (ZPICs) Quality Improvement Organizations (QIOs) Redetermination Once contractor makes an initial determination, a provider has 120 days to file a request for redetermination Request for redetermination must be filed within 30 days after the date of the first demand letter in order to avoid recoupment of the overpayment. Recoupment begins on the 41st st day after the date of the demand letter. The contractor has 60 days from the date of the redetermination request to issue a decision Providers may submit additional evidence after the request is submitted, and the contractor may extend the 60 day decision decision‐making time period by 14 days for each submission. Reconsideration Once the contractor issues a reconsideration decision, a provider has 180 days to file a request for reconsideration. Request for reconsideration must be filed within 60 days after the redetermination decision in order to avoid recoupment of the overpayment. Recoupment begins on the 76 Administrative Law Judge (ALJ) Hearing A provider must file a request for an ALJ hearing within 60 days of the QIC’s reconsideration decision. Amount in controversy requirement must be met ALJ hearing may be conducted in person, by video teleconference (VTC), or by phone CMS will recoup the alleged overpayment during this and following stages of appeal Consideration: MAC reversal of favorable findings by ALJ Administrative Law Judge (ALJ) Hearing Discovery versus requests for information Discovery is only permitted when CMS or its contractors participate as a party Regardless, providers can submit FOIA requests for information and request to review the audit file CMS or its contractors may participate in the hearing without being a party Medicare Appeals Council (MAC) A provider dissatisfied with the ALJ decision has 60 days to file an appeal to the Medicare Appeals Council (MAC) Federal District Court A provider must submit an appeal to the federal district court within 60 days of the date of the MAC decision amount in controversy requirements must be met.

Medicare Program Integrity Manual Chapter 8 – Administrative Actions and Statistical

Sampling for Overpayment Estimates

Table of Contents (Rev. 377, 05-27-11)

Transmittals for Chapter 8

8.1 – Appeal of Denials 8.2 – Overpayment Procedures

8.2.1 – Overpayment Assessment Procedures 8.2.1.1 – Definition of Overpayment Assessment Terms

8.2.2 – Assessing Overpayment When Review Was Based on Statistical Sampling for Overpayment Estimation 8.2.3 – Assessing Overpayment or Potential Overpayment When Review Was Based on Limited Sample or Limited Sub–sample

8.2.3.1 – Contractor Activities to Support Assessing Overpayment 8.2.3.2 – Conduct of Expanded Review Based on Statistical Sampling for Overpayment Estimation and Recoupment of Projected Overpayment by Contractors 8.2.3.3 - Consent Settlement Instructions

8.2.3.3.1 - Background on Consent Settlement 8.2.3.3.2 - Opportunity to Submit Additional Information Before Consent Settlement Offer 8.2.3.3.3 - Consent Settlement Offer 8.2.3.3.4 - Option 1 - Election to Proceed to Statistical Sampling for Overpayment Estimation 8.2.3.3.5 - Option 2 - Acceptance of Consent Settlement Offer 8.2.3.3.6 - Consent Settlement Budget and Performance Requirements for ACs

8.2.4 – Coordination with Audit and Reimbursement Staff 8.3 – Suspension of Payment

8.3.1 – When Suspension of Payment May Be Used 8.3.1.1 – Fraud or Willful Misrepresentation Exists – Fraud Suspensions 8.3.1.2 – Overpayment Exists But the Amount is Not Determined – General Suspensions

8.3.1.3 – Payments to be Made May Not be Correct – General Suspensions 8.3.1.4 – Provider Fails to Furnish Records and Other Requested Information – General Suspensions

8.3.2 – Procedures for Implementing Suspension of Payment 8.3.2.1 – CMS Approval 8.3.2.2 – The Notice of Intent to Suspend

8.3.2.2.1 – Prior Notice Versus Concurrent Notice 8.3.2.2.2 – Content of Notice 8.3.2.2.3 – Shortening the Notice Period for Cause 8.3.2.2.4 – Mailing the Notice to the Provider 8.3.2.2.5 – Opportunity for Rebuttal

8.3.2.3 – Claims Review During the Suspension Period 8.3.2.3.1 – Claims Review 8.3.2.3.2 – Case Development - Benefit Integrity

8.3.2.4 – Duration of Suspension of Payment 8.3.2.5 – Removing the Suspension 8.3.2.6 – Disposition of the Suspension 8.3.2.7 – Contractor Suspects Additional Improper Claims

8.3.3 – Suspension Process for Multi–Region Issues 8.3.3.1 – DME MACs and DME PSCs and ZPICs 8.3.3.2 – Reserved for Future Use

8.4 – Use of Statistical Sampling for Overpayment Estimation 8.4.1 – Introduction

8.4.1.1 – General Purpose 8.4.1.2 - The Purpose of Statistical Sampling 8.4.1.3 – Steps for Conducting Statistical Sampling 8.4.1.4 - Determining When Statistical Sampling May be Used 8.4.1.5 – Consultation With a Statistical Expert 8.4.1.6 – Use of Other Sampling Methodologies

8.4.2 – Probability Sampling 8.4.3 – Selection of Period to be Reviewed and Composition of Universe

8.4.3.1 – Selection of Period for Review 8.4.3.2 – Defining the Universe, the Sampling Unit, and the Sampling Frame

8.4.3.2.1 – Composition of the Universe 8.4.3.2.2 – The Sampling Unit 8.4.3.2.3 – The Sampling Frame

8.4.4 – Sample Selection

8.4.4.1 – Sample Design 8.4.4.1.1 – Simple Random Sampling 8.4.4.1.2 – Systematic Sampling 8.4.4.1.3 – Stratified Sampling 8.4.4.1.4 – Cluster Sampling 8.4.4.1.5 – Design Combinations

8.4.4.2 – Random Number Selection 8.4.4.3 – Determining Sample Size 8.4.4.4 – Documentation of Sampling Methodology

8.4.4.4.1 – Documentation of Universe and Frame 8.4.4.4.2 – Arrangement and Control Totals 8.4.4.4.3 – Worksheets 8.4.4.4.4 – Overpayment/Underpayment Worksheets

8.4.4.5 – Informational Copies to Primary GTL, Associate GTL, SME or CMS RO

8.4.5 – Calculating the Estimated Overpayment 8.4.5.1 – The Point Estimate 8.4.5.2 – Calculation of the Estimated Overpayment Amount

8.4.6 – Actions to be Performed Following Selection of Provider or Supplier and Sample

8.4.6.1 – Notification of Provider or Supplier of the Review and Selection of the Review Site

8.4.6.1.1 – Written Notification of Review 8.4.6.1.2 – Determining Review Site

8.4.6.2 – Meetings to Start and End the Review 8.4.6.3 – Conducting the Review

8.4.7- Overpayment Recovery 8.4.7.1 – Recovery from Provider or Supplier 8.4.7.2 – Informational Copy to Primary GTL, Associate GTL, SME or CMS RO

8.4.8 – Corrective Actions 8.4.9 – Changes Resulting from Appeals

8.4.9.1 – Sampling Methodology Overturned 8.4.9.2 – Revised Initial Determination

8.4.10 – Resources 8.4.11 – Additional Discussion of Stratified Sampling and Cluster Sampling

8.4.11.1 – Stratified Sampling 8.4.11.2 – Cluster Sampling

8.1 - Appeal of Denials (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) A claimant dissatisfied with a contractor’s initial determination is entitled by law and regulations to specified appeals. The appeals process allows a provider and/or a beneficiary (or representative) the right to request a review or reconsideration of the determination to deny a service in full or in part. In this process, Hearing Officers (HOs) and ALJs look to the evidence of record and must base their decision upon a preponderance of the evidence. If the appeal is of a claim reviewed by a PSC, then the PSC forwards its records on the case to the AC so that it can handle the appeal. As conclusory statements may be considered of little or questionable value, it is important that reviewers include clearly articulated rationale for their findings. Such clearly articulated rationale will continue to be of importance if a denial is appealed beyond the ALJ level to the Appeals Council or eventually to federal court. Contractors must include a copy of the policy underlying denial in the case file. A. Use of Medical Specialist Reviewers may also use medical specialists to lend more weight and credibility to their rationale or findings. When an adjudicator must weigh the statements and rationale furnished by the appellant provider against the statements and rationale of the reviewer (and any information used by the reviewer), the opinion of a specialist in the same area as the provider may carry greater weight than the opinion of a non-specialist. Consequently, PSCs are required to have a medical specialist involved in denials that are not based on the application of clearly articulated policy with clearly articulated rationale. A review or reconsideration involving the use of medical judgment should involve consultation with a medical specialist. Additionally, contractors are encouraged to use specialists whenever possible since providers are more likely to accept the opinion (and any resulting overpayment) of a specialist in their own area. B. Documenting Reopening and Good Cause Reopening occurs when a PSC conducts a review of claims at any time after the initial/review determination (see 42 CFR 405.980, (b).) If reopening and conducting a postpayment review occurs within 12 months of the initial/review determination, the PSC does not need to establish good cause. However, the PSC should document the date so there is no confusion about whether good cause should have been established. After 12 months, but within 4 years from the date of the initial/review determination, contractors must establish good cause. (See Medicare Claims Processing Manual Pub 100-04, chapter 34 and 42 CFR 405.986. Documenting the date a claim was reopened (regardless of the demand letter issue date) and the rationale for good cause when claims are reopened more than 12 months from the initial/review determination will lend credibility to contractor documentation if the determination is appealed.

8.2 – Overpayment Procedures (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The PSCs and the ZPICs shall refer all identified overpayments to the AC or MAC who shall send the demand letter and recoup the overpayment. Contractors should initiate recovery of overpayments whenever it is determined that Medicare has erroneously paid. In any case involving an overpayment, even where there is a strong likelihood of fraud, request recovery of the overpayment. PSC or ZPIC BI units shall notify law enforcement of their intention to collect outstanding overpayments in cases in which they are aware of a pending investigation. There may be situations where OIG/OI or other law enforcement agencies might recommend that overpayments are postponed or not collected; however, this must be made on a case-by-case basis, and only when recovery of the overpayment would undermine the specific law enforcement actions planned or currently taking place. PSCs or ZPICs shall refer such requests to the Primary GTL, Associate GTL, and SME. If delaying recoupment minimizes eventual recovery, delay may not be appropriate. PSCs or ZPICs shall forward any correspondence received from law enforcement requesting the overpayment not be recovered to the Primary GTL, Associate GTL, and SME. The Primary GTL, Associate GTL, and SME will decide whether or not to recover. If a large number of claims are involved, contractors consider using statistical sampling for overpayment estimation to calculate the amount of the overpayment. (See PIM, chapter 8, §8.4.) Contractors have the option to request the periodic production of records or supporting documentation for a limited sample of submitted claims from providers or suppliers to which amounts were previously overpaid to ensure that the practice leading to the overpayment is not continuing. The contractor may take any appropriate remedial action described in this chapter if a provider or supplier continues to have a high level of payment error. Offer the provider a consent settlement based on the potential projected overpayment amount. 8.2.1 – Overpayment Assessment Procedures (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11)) After an overpayment determination is made concluding an incorrect amount of money has been paid, contractors must assess an overpayment. The assessment options vary depending upon the type of sample used when identifying beneficiary claims for inclusion in the postpay review. Whenever possible, CMS encourages contractors to report postpayment savings in terms of:

• Actual overpayment;

• Settlement based overpayment, or

• Statistically extrapolated overpayments. A. Example Format of An Overpayment Worksheet (also see Exhibit 46)

Provider Name

Provider UPIN or PIN:

Reason for Review

Type of Sample Reviewed: Statistical Sampling for Overpayment Estimation

Explanation of Sampling Methodology:

Number of Claims in Sample:

Number of Claims in Universe:

Amount of Overpayment (after allowance for deductible and coinsurance)

Claims Reviewed

Billed Amount

Allowed Amount

Rationale for Denial

§1879 Determinations

§1870 Determinations

Total Actual Overpayment

Overpayment extrapolated over the universe

8.2.1.1 – Definition of Overpayment Assessment Terms (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11)) A. Actual Overpayment

An actual overpayment is, for those claims reviewed, the sum of payments (based on the amount paid to the provider and Medicare approved amounts) made to a provider for services which were determined to be medically unnecessary or incorrectly billed. B. Projected Overpayment A projected overpayment is the numeric overpayment obtained by projecting an overpayment from statistical sampling for overpayment estimation to all similar claims in the universe under review. C. Limited Projected Overpayment A limited projected overpayment is the numeric overpayment obtained by projecting an overpayment from a limited sample or limited sub-sample to all similar claims in the universe under review. 8.2.2 – Assessing Overpayment When Review Was Based on Statistical Sampling for Overpayment Estimation (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) If contractors use statistical sampling for overpayment estimation of claims, they follow instructions in Chapter 3, §3.10 to calculate the valid projected overpayment. They document the sampling methodology when review is based on statistical sampling for overpayment estimation. They notify the provider of the overpayment and refer the case to overpayment staff to make payment arrangements with the provider to collect the overpayment. 8.2.3 – Assessing Overpayment or Potential Overpayment When Review Was Based on Limited Sample or Limited Sub-sample (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) If a limited sample or limited sub-sample of claims is chosen for review, there are three overpayment assessment options for contractors:

• Refer to overpayment staff for recoupment of the actual overpayment for the claims reviewed;

• Conduct an expanded review based on statistical sampling for overpayment

estimation instructions in Chapter 8, §8.4 and recoup the projected overpayment; or

• Offer the provider a consent settlement based on the potential projected

overpayment amount. 8.2.3.1 – Contractor Activities to Support Assessing Overpayment (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11)

A. Step 1 The first step in assessing an overpayment is for contractors to document for each claim reviewed the following:

• The amount of the original claim; • The allowed amount;

• The rationale for denial;

• The §1879 determination for each assigned claim in the sample denied because

the service was not medically reasonable and necessary (or the §1842(1) provider refund determination on non-assigned provider claims denied on the basis of §1862 (a)(1)(A)) (see PIM Chapter 3 §3.6.7 and Exhibit 14.1);

• The §1870 determination for the provider for each overpaid assigned claim in the

sample (see PIM Chapter 3 §3.6.7 and Exhibit 14.2); and

• The amount of overpayment (after allowance for deductible and coinsurance). B. Step 2 Notify the provider of the preliminary overpayment findings and preliminary review findings. C. Step 3 If the provider submits additional documentation, review the material and adjust the preliminary overpayment findings, accordingly. D. Step 4 Calculate the final overpayment. E. Step 5 Refer to the overpayment recoupment staff. 8.2.3.2 – Conduct of Expanded Review Based on Statistical Sampling for Overpayment Estimation and Recoupment of Projected Overpayment by Contractors (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11)

The ACs and MACs shall perform the actual recoupment identified by the PSCs or the ZPICs. A. If an expanded review of claims is conducted, contractors shall follow the sampling instructions found in PIM chapter 8, §8.4 obtain and review claims and medical records, and document for each claim reviewed:

ο The amount of the original claim;

ο The allowed amount;

ο The rationale for denial;

o The §1879 determination for each assigned claim in the sample denied because the service was not medically reasonable and necessary (or the §1842(1) provider refund determination on non-assigned provider claims denied on the basis of §1862(a)(1)(A)) (see PIM chapter 3, §3.6.7 and exhibit 14.1);

ο The §1870 determination for the provider for each overpaid assigned claim in the sample (see PIM chapter 3, §3.6.7 and exhibit 14.2); and

ο The amount of overpayment (after allowance for deductible and coinsurance). B. Contractors calculate the projected overpayment by extrapolating from the actual overpayment to the universe that excludes those claims determined that the provider did not have knowledge that the service was not medically necessary; C. Notify the provider of the preliminary projected overpayment findings and review findings; D. If the provider submits additional documentation, review the material and adjust the preliminary projected overpayment findings, accordingly; E. Calculate the final overpayment; and F. Refer to the overpayment recoupment staff. 8.2.3.3 - Consent Settlement Instructions (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) 8.2.3.3.1 - Background on Consent Settlement (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The Medicare Prescription Drug, Improvement, and Modernization Act (MMA) of 2003 defines consent settlement as an agreement between the Secretary and a provider of services or supplier whereby both parties agree to settle a projected overpayment based

on less than a statistically valid sample of claims and the provider of services or supplier agrees not to appeal the claims involved. The PSC and ZPIC BI units and the contractor medical review units shall submit via secure email the consent settlement to the Primary and Associate GTLs before offering a consent settlement to the provider or supplier. If the PSC or the ZPIC BI units or the contractor medical review units do not have secure email, the consent settlement shall be sent to the Primary GTL and the Associate GTL via hard copy. Upon receipt, GTLs will forward the consent settlement to the Director of the Division of Benefit Integrity Management Operations. The PSC or the ZPIC BI units and the contractor medical review units may contact the provider upon approval of the consent settlement. Consent settlement documents carefully explain, in a neutral tone, what rights a provider waives by accepting a consent settlement. The documents shall also explain in a neutral tone the consequences of not accepting a consent settlement. A key feature of a consent settlement is a binding statement that the provider agrees to waive any rights to appeal the decision regarding the potential overpayment. The consent settlement agreement shall carefully explain this, to ensure that the provider is knowingly and intentionally agreeing to a waiver of rights. Consent settlement correspondence shall contain: A complete explanation of the review and the review findings A thorough discussion of §1879 and §1870 determinations, where applicable The consequences of deciding to accept or decline the consent settlement offer It is rare that a PSC or ZPIC BI unit will offer and develop a consent settlement. However, when the PSC or ZPIC offers and develops a consent settlement, the AC or MAC shall administer the settlement. 8.2.3.3.2 - Opportunity to Submit Additional Information Before Consent Settlement Offer (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The Medicare Prescription Drug, Improvement, and Modernization Act of 2003, section 935(a)(5) states the provider has the opportunity to submit additional information before being offered a consent settlement. Based on a postpayment review of the medical records, the contractor shall communicate in writing to the provider or supplier that:

• The preliminary evaluation of the records indicates there would be an overpayment;

• The nature of the problems in the billing and practice patterns identified in the

evaluation; • The steps that the provider or supplier can take to address the problems; and

• The provider or supplier has forty-five (45) days to furnish additional information concerning the medical records for the claims that have been reviewed.

If after forty-five (45) days, it is determined that there is still an overpayment, then the provider or supplier shall receive a consent settlement offer. If an overpayment is not warranted after additional review, then a follow-up letter shall be sent to the provider or supplier stating that no additional action is deemed necessary. 8.2.3.3.3 - Consent Settlement Offer (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) After the additional information concerning the medical records for the claims reviewed have been assessed and if it is still determined that there was an overpayment, the contractor shall offer the provider or supplier the opportunity to proceed with statistical sampling for overpayment estimation or a consent settlement. The PSC or the ZPIC BI units and the contractor medical review units may choose to present the consent settlement letter to the provider or supplier in a face-to-face meeting. The consent settlement correspondence shall describe the two options available to the provider or supplier. The provider or supplier is given 60 days from the date of the correspondence to choose an option. If there is no response, Option 1 shall be selected by default. 8.2.3.3.4 - Option 1 - Election to Proceed to Statistical Sampling for Overpayment Estimation (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) If a provider or supplier fails to respond, this option shall be selected by default. For providers or suppliers who select this option knowingly or by default, thereby rejecting the consent settlement offer and retaining their full appeal rights, PSC BI units and the contractor medical review units shall;

• Notify the provider or supplier of the actual overpayment and refer to overpayment recoupment staff; and

• Initiate statistical sampling for overpayment estimation of the provider's or supplier’s claims for the service under review following instructions in the Program Integrity Manual, chapter 8, §8.4 If the review results in a decision to recoup the overpayment, the overpayment collection shall be initiated within 12 months of the decision. 8.2.3.3.5 - Option 2 - Acceptance of Consent Settlement Offer (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) A provider or supplier accepting Option 2 waives any appeal rights with respect to the alleged overpayment. Providers or suppliers selecting Option 2 that have any additional claims shall not be audited for the service under review within the same time period.

Model language for the consent settlement documents can be found in PIM Exhibit 15. 8.2.3.3.6 - Consent Settlement Budget and Per formance Requirements for ACs (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) When supporting PSCs or ZPICs in consent settlements, the ACs shall report these costs in the PSC support activity code 23201. 8.2.4 - Coordination With Audit and Reimbursement Staff (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Intermediary MR staff must work closely with their Audit/Reimbursement staff from the beginning of the postpay process to ensure that the universe selected is appropriate and that overpayments and underpayments are accurately determined and reflected on the provider's cost report. They furnish the Audit/Reimbursement staff the following information upon completion of the postpayment review:

• The sample documentation contained in the PIM Chapter 3, §3.6.3; • The identification of incorrectly paid or incorrectly denied services; and

• All other information required by the Cost Report Worksheets in PIM Chapter 3,

§3.6.1 and applicable Exhibits. They also furnish the above information if adjustments are made as a result of appeals. In most instances, the Audit/Reimbursement staff will:

• Determine the overpayment to be recovered based on MR findings and pursue the recovery of the overpayment; and

• Use the information MR provides on their postpayment review findings to ensure

an accurate settlement of the cost report and/or any adjustments to interim rates that may be necessary as a result of the MR findings. To preserve the integrity of Provider Statistical and Reimbursement Report (PS&R) data relative to paid claims and shared systems data relative to denied claims, and to ensure proper settlement of costs on provider cost reports, the same data must be used when the projection is made as was used when the sample was selected. Individual claims will not be adjusted. In the event that a cost report has been settled, Audit/Reimbursement staff will determine the impact on the settled cost report and the actions to be taken.

Projections on denied services must be made for each discipline and revenue center when PPS is not the payment method.

When notifying the provider of the review results for cost reimbursed services, MR must explain that the stated overpayment amount represents an interim payment adjustment. Indicate that subsequent adjustments may be made at cost report settlement to reflect final settled costs. Information from the completed Worksheets 1 - 7 must be routed to the Audit and Reimbursement staff. In addition to the actual and projected overpayment amounts, the information must provide the number of denied services (actual denied services plus projected denied services) for each discipline and the amounts of denied charges (actual denied amounts plus projected denied amounts) for supplies and drugs. Upon completion of the review, furnish the Audit and Reimbursement staff with the information listed in the PIM. 8.3 – Suspension of Payment (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The process by which the PSC or ZPIC notifies and coordinates with the AC or MAC of a CMS-approved suspension of payment shall be documented in the JOA. PSCs and ZPICs shall advise and coordinate with the AC or MAC when payment suspension has been approved by CMS. The PSCs and ZPICs shall perform the necessary medical review for suspensions for which they have recommended and received CMS approval. Medicare authority to withhold payment in whole or in part for claims otherwise determined to be payable is found in federal regulations at 42 CFR 405.370-377, which provides for the suspension of payments. 8.3.1 – When Suspension of Payment May Be Used (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Suspension may be used when there is reliable information that:

• Fraud or willful misrepresentation exists; • An overpayment exists but the amount of the overpayment is not yet determined;

• The payments to be made may not be correct; or

• The provider fails to furnish records and other requested information needed to

determine the amounts due the provider or supplier. These four reasons for implementing a suspension of payment are described more fully below.

NOTE: For providers that file cost reports, suspension may have little impact. If the provider is receiving periodic interim payments (PIP), interim payments may be suspended. If the provider is not on PIP, suspension will affect the settlement of the cost report. When an overpayment is determined, the amount is not included in any settlement amount on the cost report. For example, if the intermediary has suspended $100,000, when the cost report is settled, the intermediary would continue to hold the $100,000. This means if the cost report shows CMS owing the provider $150,000, the provider would only receive $50,000 until the suspension action has been completed. If the provider owes CMS money at settlement, the amount of the suspended payment would increase the amount owed by the provider. In most instances, intermediaries should adjust interim payments to reflect projected cost reductions. Limit the adjustment to the percentage of potential fraud or the total payable amount for any other reasons. For example, if the potential fraud involved 5 percent of the interim rate, the reduction in payment is not to exceed 5 percent. Occasionally, suspension of all interim payments may be appropriate. 8.3.1.1 – Fraud or Willful Misrepresentation Exists - Fraud Suspensions (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Suspension of payment may be used when the contractor, MAC, PSC or ZPIC or CMS possesses reliable information that fraud or willful misrepresentation exists. For the purposes of this section, these types of suspensions will be called “fraud suspensions.” Fraud suspensions may also be imposed for reasons not typically viewed within the context of false claims. An intermediary example is that the QIO has reviewed inpatient claims and determined that the diagnosis related groups (DRGs) have been upcoded. As an example, contractors or MACs may find is that suspected violation of the physician self referral ban is cause for suspension since claims submitted in violation of this statutory provision must be denied and any payment made would constitute an overpayment. Forged signatures on Certificates of Medical Necessity (CMN), treatment plans, and other misrepresentations on Medicare claims and claim forms to obtain payment result in overpayments. Credible allegations of such practices are cause for suspension pending further development. Whether or not the contractor, MAC, PSC or ZPIC recommends suspension action to CMS is a case-by-case decision requiring review and analysis of the allegation and/or facts. The following information is provided to assist the contractor, MAC, PSC or ZPIC in deciding when to recommend suspension action. A. Complaints There is considerable latitude with regard to complaints alleging fraud and abuse. The history, or newness of the provider, the volume and frequency of complaints concerning the provider, and the nature of the complaints all contribute to whether suspension of payment should be recommended. If there is a credible allegation(s) that a provider is submitting or may have submitted false claims, the contractor, MAC, PSC or ZPIC shall

recommend suspension of payment to the CMS Central Office (CO) Division of Benefit Integrity Management Operations Fraud and Abuse Suspensions and Sanctions (DBIMO FASS) team. B. Provider Identified in CMS Fraud Alert Contractors, MACs, PSCs and ZPICs shall recommend suspension to the CO DBIMO FASS team if a provider in their jurisdiction is the subject of a CMS national Fraud Alert and the provider is billing the identical items/services cited in the alert or if payment for other claims must be suspended to protect the interests of the government. C. Requests from Outside Agencies Contractors, MACs, PSCs, and ZPICs shall follow the suspension of payment actions for each agency request indicated below.

• CMS -- Initiate suspension as requested. • OIG/FBI – Contractors, MACs, PSCs, and ZPICs shall forward the written

request to the CO DBIMO FASS team for its review and determination. The CO DBIMO FASS team will decide.

• AUSA/DOJ – Contractors, MACs, PSCs, and ZPICs shall forward the written

request to the CO DBIMO FASS team for review and determination.

• Other – Other situations the contractor, MAC, PSC or ZPIC may consider recommending suspension of payment to the CO DBIMO FASS team are:

o Provider has pled guilty to, or been convicted of, Medicare, Medicaid,

CHAMPUS, or private health care fraud and is still billing Medicare for services;

o Federal/State law enforcement has subpoenaed the records of, or executed

a search warrant at, a health care provider billing Medicare; o Provider has been indicted by a Federal Grand Jury for fraud, theft,

embezzlement, breach of fiduciary responsibility, or other misconduct related to a health care program;

o Provider presents a pattern of evidence of known false documentation

or statements sent to the contractor or the MAC; e.g., false treatment plans, false statements on provider application forms.

8.3.1.2 – Overpayment Exists But the Amount is Not Determined - General Suspensions (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11)

Suspension of payment may be used when the contractor, MAC, PSC or ZPIC or CMS possesses reliable information that an overpayment exists but has not yet determined the amount of the overpayment. In this situation, the contractor, MAC, PSC, and ZPIC shall recommend suspension to the CO DBIMO FASS team. For the purposes of this section, these types of suspensions will be called “general suspensions.” EXAMPLE: Several claims identified on post-pay review were determined to be non-covered or miscoded. The provider has billed this service many times before and it is suspected that there may be a number of additional non-covered or miscoded claims that have been paid. 8.3.1.3 – Payments to be Made May Not be Correct - General Suspensions (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Suspension of payment may be used when the contractor, MAC, PSC or ZPIC or CMS possesses reliable information that the payments to be made may not be correct. In this situation, the contractor, MAC, PSC, and ZPIC shall recommend suspension to the CO DBIMO FASS team. For the purposes of this section, these types of suspensions will be called “general suspensions”. 8.3.1.4 – Provider Fails to Furnish Records and Other Requested Information - General Suspensions (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Suspension of payment may be used when the contractor, MAC, PSC or ZPIC or CMS possesses reliable information that the provider has failed to furnish records and other information requested or that is due, and which is needed to determine the amounts due the provider. In this situation, the contractor, MAC, PSC, and ZPIC shall recommend suspension to the CO DBIMO FASS team. For the purposes of this section, these types of suspensions will be called “general suspensions”. EXAMPLE: During a postpayment review, medical records and other supporting documentation are solicited from the provider to support payment. The provider fails to submit the requested records. The contractor determines that the provider is continuing to submit claims for services in question. 8.3.2 – Procedures for Implementing Suspension of Payment (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) 8.3.2.1 – CMS Approval (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11)

The initiation (including whether or not to give advance notice), modification, or removal of any type of suspension requires the explicit prior approval of the CMS CO DBIMO FASS team. The contractor, MAC, PSC, ZPIC or the CO DBIMO FASS team will coordinate suspension action with law enforcement partners. The contractor, MAC, PSC or ZPIC shall forward a draft of the proposed notice of suspension and a brief summary of the evidence upon which the recommendation is based to the CO DBIMO FASS team. The contractor, MAC, PSC, and ZPIC shall not take suspension action without the explicit approval of the CO DBIMO FASS team. In most cases, the PSC or ZPIC will notify OIG and other law enforcement partners of its decision and will keep law enforcement apprised of any future decisions to modify the suspension. However, if a contractor, MAC, PSC or ZPIC, or CMS has been working with law enforcement on the case, immediately notify them of the proposed recommendation being submitted to the CO DBIMO FASS team. Notice may consist of a telephone call or a fax. If law enforcement wants more time to study or discuss the suspension, contractors, MACs, PSCs, and ZPICs shall discuss their request with the CO DBIMO FASS team. If law enforcement requests that suspension action should, or should not, be taken, contractors, PSCs, and ZPICs shall contact the CO DBIMO FASS team. Contractors, MACs, PSCs and ZPICs shall also advise law enforcement that the request must be in writing and must provide a detailed rationale justifying why payment should, or should not, be suspended. 8.3.2.2 – The Notice of Intent to Suspend (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) 8.3.2.2.1 – Prior Notice Versus Concurrent Notice (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Contractors, MACs, PSCs, and ZPICs shall inform the provider of the suspension action being taken. When prior notice is appropriate, give at least 15 calendar days prior notice. Day one begins the day after the notice is mailed. A. Medicare Trust Fund would be harmed by giving prior notice: Contractors, MACs, PSCs or ZPICs shall recommend to the CO DBIMO FASS team, not to give prior notice if in the contractor’s, MAC’s, PSC’s or ZPIC’s opinion, any of the following apply: 1. Delay in suspension will cause the overpayment to rise at an accelerated rate (i.e., dumping of claims);

2. There is reason to believe that the provider may flee the contractor’s or MAC’s jurisdiction before the overpayment can be recovered; or 3. The contractor, MAC, PSC or ZPIC has first hand knowledge of a risk that the provider will cease or severely curtail operations or otherwise seriously jeopardize its ability to repay its debts.

If the CO DBIMO FASS team waives the advance notice requirement, contractors, MACs, PSCs and ZPICs shall send the provider notice concurrent with implementation of the suspension, but no later than 15 days, after suspension is imposed. B. Suspension imposed for failure to furnish requested information: Contractors, MACs, PSCs or ZPICs shall recommend that the CO DBIMO FASS team waive prior notice requirements for failure to furnish information requested by the contractor, MAC, PSC or ZPIC that is needed to determine the amounts due the provider. If the CO DBIMO FASS team waives the prior notice requirement, contractors, MACs, PSCs and ZPICs shall send the provider notice concurrent with implementation of the suspension, but no later than 15 days after the suspension is imposed. C. Fraud suspension: With respect to fraud suspensions, contractors, MACs, PSCs and ZPICs shall recommend to the CO DBIMO FASS team that prior notice not be given. The CO DBIMO FASS team will decide whether to waive the notice. The CO DBIMO FASS team will also direct the content of the notice. If the CO DBIMO FASS team waives the advance notice requirement, the contractor, MAC, PSC or ZPIC shall send the provider notice concurrent with implementation of the suspension, but no later than 15 days, after suspension is imposed. 8.3.2.2.2 – Content of Notice (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Contractors, MACs, PSCs and ZPICs shall prepare a “draft notice” and send it, along with the recommendation and any other supportive information, to the CO DBIMO FASS team for approval. The draft notice shall include, at a minimum:

• That suspension action will be imposed; • The extent of the suspension (i.e., all claims, certain types of claims, 100 percent

suspension or partial suspension); • That suspension action is not appealable; • That CMS has approved implementation of the suspension; • When suspension will begin; • The items or services affected; • How long the suspension is expected to be in effect; • The reason for suspending payment;

• That the provider has the opportunity to submit a rebuttal statement within 15

days of notification; and • Where to mail the rebuttal.

In the notice, contractors, MACs, PSCs and ZPICs shall also state why the suspension action is being taken. For fraud suspensions, the contractor, MAC, PSC or ZPIC shall do so in a way that does not disclose information that would undermine a potential fraud case. The rationale must be specific enough to justify the action being taken and allow the provider an opportunity to identify the problem. The CO DBIMO FASS team will direct the content of the notice. The notice does not need to specify that the provider is suspected of fraud or willful misrepresentation. The notice shall include a limited selection of claims received that indicate payment may not have been collected. 8.3.2.2.3 – Shortening the Notice Period for Cause (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) At any time, the contractor, MAC, PSC or ZPIC may recommend to the CO DBIMO FASS team that the advance notice be shortened during the notice period. Such a recommendation would be appropriate if the contractor, MAC, PSC or ZPIC believes that the provider is intentionally submitting additional claims in anticipation of the effective date of the suspension. If suspension is imposed earlier than indicated in the notice, the contractor, MAC, PSC or ZPIC shall notify the provider in writing of the change and the reason. 8.3.2.2.4 – Mailing the Notice to the Provider (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) After consultation with and approval from the CO DBIMO FASS team, contractors, MACs, PSCs and ZPICs shall send the notice of suspension to the provider. In the case of fraud suspensions, they send a copy to the OIG, FBI, or AUSA if they have been previously involved. 8.3.2.2.5 – Opportunity for Rebuttal (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The suspension notice gives the provider an opportunity to submit to the contractor, MAC, PSC or ZPIC a statement within 15 days indicating why suspension action should not be, or should not have been, imposed. However, this may be shortened or lengthened for cause (see 42 CFR 405.374(b)). A provider’s reaction to suspension may include threats of court action to restore payment or to stop the proposed action. The CO DBIMO FASS team will consult with OGC and will advise the contractor, MAC, PSC or ZPIC before the contractor, MAC, PSC or ZPIC responds to any rebuttal statements.

Contractors, MACs, PSCs and ZPICs shall ensure the following:

• CMS Review – Contractors, MACs, PSCs and ZPICs shall immediately forward provider responses and a draft response to the CMS CO DBIMO FASS team.

• Timing – Implementation of suspension actions is not delayed by the receipt

and/or review of the rebuttal statement. The suspension goes into effect as indicated in the notice.

• Review of Rebuttal – Because suspension actions are not appealable, the rebuttal

is the provider’s only opportunity to present information as to why suspension action should be non-initiated or terminated. Contractors, MACs, PSCs and ZPICs shall also carefully review the provider’s rebuttal statement and consider all facts and issues raised by the provider. If the contractor, MAC, PSC or ZPIC is convinced that the suspension action should be non-initiated or terminated, they shall consult immediately with the CO DBIMO FASS team.

• Response – Respond to the provider’s rebuttal within 15 days from the date the

statement is received, following consultation and approval from the CO DBIMO FASS team.

8.3.2.3 – Claims Review During the Suspension Period (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) 8.3.2.3.1 – Claims Review (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) A. Claims Review of Suspended Claims: Once suspension has been imposed, contractors, MACs, PSCs and ZPICs shall follow normal claims processing and MR procedures. Contractors and MACs shall make every attempt within the MR budget to determine if suspended claims are payable. Contractors, MACs, PSCs and ZPICs shall ensure that the provider is not substituting a new category of improper billing to counteract the effect of the payment suspension. If the claim is determined to be not payable, it shall be denied. For claims that are not denied, the contractor or MAC shall send a remittance advice to the provider showing that payment was approved but not sent. Contractors, MACs, PSCs and ZPICs are not required to perform 100 percent pre-pay medical review of suspended claims. If 100 percent prepayment review is not conducted, a 100 percent postpayment review shall be performed on all claims adjudicated during the suspension, prior to the issuance of the overpayment determination. Contractors, MACs, PSCs and ZPICs shall consult with the CO DBIMO FASS team when resources may be better utilized employing statistical sampling procedures. Contractors, MACs, PSCs and ZPICs shall use the principles of statistical sampling found in the PIM, Chapter 8, §8.4, to determine what percentage of claims in a given universe of suspended claims are payable.

B. Review of Suspected Fraudulent or Overpaid Claims: Contractors, MACs, PSCs and ZPICs shall follow procedures in the PIM Chapter 3, §3.8 in establishing an overpayment. The overpayment consists of all claims in a specific time period determined to have been paid incorrectly. Contractors, MACs, PSCs and ZPICs shall make all reasonable efforts to expedite the determination of the overpayment amount. NOTE: Claims selected for postpayment review may be reopened within 1 year for any reason or within 4 years for good cause. Cost report determinations may be reopened within 3 years after the Notice of Program Reimbursement has been issued. Good cause is defined as new and material evidence, error on the face of the record, or clerical error. The regulations have open-ended potential for fraud or similar fault. The exception to the 1-year rule is for adjustments to DRG claims. A provider has 60 days to request a change in an assignment of a DRG. (See 42 CFR 412.60(d).) 8.3.2.3.2 – Case Development – Benefit Integrity (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Even though suspension action was recommended and/or implemented, PSCs and ZPICs shall discuss the case with the OIG to ascertain their interest in working the case. If OIG declines the case, they shall discuss whether OIG referral to another law enforcement agency is appropriate. If law enforcement is not interested in the case, PSCs and ZPICs shall consider preparing the case for CMP or permissive exclusion. See PIM Chapter 4 §4.22. Whether the case is accepted by law enforcement or not, PSCs and ZPICs shall develop the overpayment as expeditiously as administratively feasible and shall keep law enforcement apprised of the dollars being withheld as well as any potential recoupment action if they are investigating the provider under suspension. The PSC and the ZPIC shall enter the suspension into the FID, no later than 5 business days after the effective date of suspension. See PIM Chapter 4, §4.11 for FID entry and update requirements. In the Suspension Narrative field, the PSC or ZPIC shall enter the items/services affected (i.e., type of item/service and applicable HCPCS/CPT codes). 8.3.2.4 – Duration of Suspension of Payment (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) A. Time Limits The CO DBIMO FASS team will initially approve suspension for a period up to 180 days. The CO DBIMO FASS team may extend the period of suspension for up to an additional 180 days upon the written request of the contractor, MAC, PSC or ZPIC, OIG, or other law enforcement agency. The request shall provide:

• Name and address of the provider under suspension;

• Amount of additional time needed (not to exceed the 180 days); and • Rationale explaining why the additional time is necessary.

B. Exceptions to Time Limits The following exceptions may apply:

• Department of Justice (including U.S. Attorneys). The CO DBIMO FASS team may grant an additional 180-day extension (beyond the first extension referred to in Section 3.9.2.4.A above) if an overpayment has not yet been determined and the Department of Justice submits a written request for an extension. Requests must include: 1) the identity of the person or entity under suspension, 2) the amount of time needed for continued suspension in order to implement an ongoing or anticipated criminal and/or civil proceeding, and 3) a statement of why and/or how criminal and/or civil actions may be affected if the suspension is not extended. This extension may be granted based on a request received by the CO DBIMO FASS team at any time before or during the period of suspension.

• OIG. The time limits in subsection A above do not apply if the case has been

referred to and is being considered by OIG for administrative sanctions (e.g., CMPs). However, this exception does not apply to pending criminal investigations by OIG. C. Provider Notice of the Extension The contractor, MAC, PSC or ZPIC shall obtain the CO DBIMO FASS team decision about the extension request, and shall notify the provider if the suspension action has been extended. 8.3.2.5 – Removing the Suspension (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Contractors, MACs, PSCs, and ZPICs shall recommend to the CO DBIMO FASS team that suspension of payments be terminated when the time limit expires. No action associated with termination shall be taken without the approval by the CO DBIMO FASS team. The contractor, MAC, PSC or ZPIC may recommend to the CO DBIMO FASS team that a suspension be terminated earlier if the basis for the suspension action was that an overpayment may exist, and the contractor, MAC, PSC, or ZPIC has determined the amount of the overpayment, if any. B. If the basis for the suspension action was that fraud or willful misrepresentation existed, there is satisfactory evidence that the fraud activity has ceased, and the amount of suspended monies exceeds the estimated amount of the suspected overpayment.

C. If the basis for the suspension action was that payments to be made may not be correct, and the contractor, MAC, PSC or ZPIC has determined that payments to be made are correct. D. If the basis for the suspension action was that the provider failed to furnish records, the provider has submitted all requested records, and the contractor, MAC, PSC or ZPIC believes the provider will comply with future requests for records. When the suspension expires or is lifted early, the disposition of the suspension shall be achieved within a reasonable time period. 8.3.2.6 – Disposition of the Suspension (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Payments for appropriate Medicare claims that are withheld during a suspension should not exceed the suspected amount of overpayment. Contractors, MACs, PSCs and ZPICs shall maintain an accurate, up-to-date record of the amount withheld and the claims that comprise the suspended amount. Contractors, MACs, PSCs and ZPICs shall keep a separate accounting of payment on all claims affected by the suspension. They shall keep track of how much money is uncontested and due the provider. The amount needs to be known as it represents assets that may be applied to reduce or eliminate any overpayment. (See PIM, chapter 8, §8.2.) Contractors, MACs, PSCs and ZPICs shall be able to provide, upon request, copies of the claims affected by the suspension. After the suspension has been removed, they shall apply the amount withheld first to the Medicare overpayment and then to reduce any other obligation to CMS or to DHHS. Contractors and MACs shall remit to the provider all monies held in excess of the amount the provider owes. If the provider owes more money than was held in suspension, the contractor or MAC shall initiate recoupment action. 8.3.2.7 – Contractor Suspects Additional Improper Claims (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) A. Present Time If the contractor, MAC, PSC or ZPIC believes that the provider will continue to submit non-covered, misrepresented, or potentially fraudulent claims, it shall consider implementing or recommending other actions as appropriate (e.g., prepayment review, a new suspension of payment.) B. Past Period of Time If the contractor, MAC, PSC or ZPIC believes there are past periods of time that may contain possible overpayments, contractors, MACs, PSCs and ZPICs shall consider recommending a new suspension of payment covering those dates. C. Additional Services

During the time that a provider is under suspension of payment for a particular service(s), if it is determined there is reason to initiate suspension action for a different service, a new suspension of payment shall be initiated or incorporated into the existing payment suspension depending on the circumstances. Anytime a new suspension action is initiated on a provider who is already under one or more suspension actions, contractors, MACs, PSCs and ZPICs shall obtain separate CMS approval, shall issue an additional notice to the provider, shall offer a new rebuttal period, etc. Model Suspension of Payment Letters can be found in Exhibit 16. 8.3.3 – Suspension Process for Multi-Region Issues (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) 8.3.3.1 – DME MACs and DME PSCs, and ZPICs (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The DME MACs, DME PSCs and ZPICs shall initiate suspension action when one of the criteria listed above is identified. (See PIM Chapter 3 §3.9.1, When Suspension of Payment May Be Used.) The following details the process that shall be followed when one DME MAC, DME PSC, or ZPIC suspends payments. A. The initiating DME MAC shall get approval from the CO DBIMO FASS team. B. The initiating DME MAC, DME PSC, or ZPIC shall share the suspension of payment information with the other DME MACs and DME PSCs and ZPICs. Reliable information that payments should be suspended in one region is sufficient reason for suspension decisions to apply to the other regions. C. The CO DBIMO FASS team will approve one suspension letter advising that payments will be held by all DME MACs and DME PSCs and ZPICs. This letter shall advise the supplier to contact the initiating DME MAC, DME PSC or ZPIC should the supplier have any questions. D. Should the suspension action require an extension of time, the CO DBIMO FASS team will approve the extension letter to the supplier. 8.3.3.2 – Reserved for Future Use (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) 8.4 - Use of Statistical Sampling for Overpayment Estimation (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) 8.4.1 – Introduction

(Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) 8.4.1.1 – General Purpose (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The purpose of this section is to provide instructions for PSC and ZPIC BI units and contractor MR units on the use of statistical sampling in their reviews to calculate and project (i.e., extrapolate) overpayment amounts to be recovered by recoupment, offset or otherwise. These instructions are provided to ensure that a statistically valid sample is drawn and that statistically valid methods are used to project an overpayment where the results of the review indicate that overpayments have been made. These guidelines are for reviews performed by the PSC or ZPIC BI units or contractor MR units. Reviews that are conducted by the PSC or ZPIC BI units or the contractor MR units to assist law enforcement with the identification, case development and/or investigation of suspected fraud or other unlawful activities may also use sampling methodologies that differ from those prescribed herein. These instructions are provided so that a sufficient process is followed when conducting statistical sampling to project overpayments. Failure by the PSC or the ZPIC BI unit or the contractor MR unit to follow one or more of the requirements contained herein does not necessarily affect the validity of the statistical sampling that was conducted or the projection of the overpayment. An appeal challenging the validity of the sampling methodology must be predicated on the actual statistical validity of the sample as drawn and conducted. Failure by the PSC or ZPIC BI units or the contractor MR units to follow one or more requirements may result in review by CMS of their performance, but should not be construed as necessarily affecting the validity of the statistical sampling and/or the projection of the overpayment. Use of statistical sampling to determine overpayments may be used in conjunction with other corrective actions, such as payment suspensions and prepayment review. 8.4.1.2 - The Purpose of Statistical Sampling (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Statistical sampling is used to calculate and project (i.e., extrapolate) the amount of overpayment(s) made on claims. The Medicare Prescription Drug, Improvement, and Modernization Act of 2003 (MMA), mandates that before using extrapolation to determine overpayment amounts to be recovered by recoupment, offset or otherwise, there must be a determination of sustained or high level of payment error, or documentation that educational intervention has failed to correct the payment error. By law, the determination that a sustained or high level of payment error exists is not subject to administrative or judicial review. 8.4.1.3 - Steps for Conducting Statistical Sampling (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11)

The major steps in conducting statistical sampling are: (1) Selecting the provider or supplier; (2) Selecting the period to be reviewed; (3) Defining the universe, the sampling unit, and the sampling frame; (4) Designing the sampling plan and selecting the sample; (5) Reviewing each of the sampling units and determining if there was an overpayment or an underpayment; and, as applicable, (6) Estimating the overpayment. Where an overpayment has been determined to exist, follow applicable instructions for notification and collection of the overpayment. 8.4.1.4 - Determining When Statistical Sampling May Be Used (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The PSC or ZPIC BI units and the contractor MR units shall use statistical sampling when it has been determined that a sustained or high level of payment error exists, or where documented educational intervention has failed to correct the payment error. A sustained or high level of payment error may be determined to exist through a variety of means, including, but not limited to:

- error rate determinations by MR unit, PSC, ZPIC or other area - probe samples - data analysis - provider/supplier history - information from law enforcement investigations - allegations of wrongdoing by current or former employees of a provider or

supplier - audits or evaluations conducted by the OIG

Once a determination has been made that statistical sampling may be used, factors also to be considered for determining when to undertake statistical sampling for overpayment estimation instead of a claim-by-claim review include, but are not limited to: the number of claims in the universe and the dollar values associated with those claims; available resources; and the cost effectiveness of the expected sampling results. 8.4.1.5 - Consultation With a Statistical Expert (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The sampling methodology used to project overpayments must be reviewed by a statistician, or by a person with equivalent expertise in probability sampling and estimation methods. This is done to ensure that a statistically valid sample is drawn and that statistically valid methods for projecting overpayments are followed. The PSC or ZPIC BI unit and the contractor MR unit shall obtain from the statistical expert a written approval of the methodology for the type of statistical sampling to be performed. If this sampling methodology is applied routinely and repeatedly, the original written approval is adequate for conducting subsequent reviews utilizing the same methodology. The PSC or ZPIC BI unit or the contractor MR unit shall have the statistical expert review the results of the sampling prior to releasing the overpayment demand letter. If questions or

issues arise during the on-going review, the PSC or ZPIC BI unit or the contractor MR unit shall also involve the statistical expert. At a minimum, the statistical expert (either on-staff or consultant) shall possess a master’s degree in statistics or have equivalent experience. See section 3.10.10 for a list, not exhaustive, of texts that represent the minimum level of understanding that the statistical expert should have. If the PSC or ZPIC BI unit or the contractor MR unit does not have staff with sufficient statistical experience as outlined here, it shall obtain such expert assistance prior to conducting statistical sampling. 8.4.1.6 - Use of Other Sampling Methodologies (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Once it is has been determined that statistical sampling may be used, nothing in these instructions precludes the Centers for Medicare & Medicaid Services (CMS) or the PSC or the ZPIC BI unit or the contractor MR unit from relying on statistically valid audit sampling methodologies employed by other law enforcement agencies, including but not limited to the OIG, the DOJ, the FBI, and other authoritative sources. Where it is foreseen that the results of a PSC or ZPIC BI unit’s or the contractor MR unit’s review may be referred to law enforcement or another agency for litigation and/or other enforcement actions, the PSC or ZPIC BI unit or the contractor MR unit shall discuss specific litigation and/or other requirements as they relate to statistical sampling with it’s statistical expert prior to undertaking the review. In addition, the PSC or ZPIC BI unit or the contractor MR unit shall discuss sampling requirements with law enforcement or other authorities before initiating the review (to ensure that the review will meet their requirements and that such work will be funded accordingly). 8.4.2 - Probability Sampling (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Regardless of the method of sample selection used, the PSC or ZPIC BI unit or the contractor MR unit shall follow a procedure that results in a probability sample. For a procedure to be classified as probability sampling the following two features must apply:

• It must be possible, in principle, to enumerate a set of distinct samples that the procedure is capable of selecting if applied to the target universe. Although only one sample will be selected, each distinct sample of the set has a known probability of selection. It is not necessary to actually carry out the enumeration or calculate the probabilities, especially if the number of possible distinct samples is large - possibly billions. It is merely meant that one could, in theory, write down the samples, the sampling units contained therein, and the probabilities if one had unlimited time; and

• Each sampling unit in each distinct possible sample must have a known

probability of selection. For statistical sampling for overpayment estimation, one

of the possible samples is selected by a random process according to which each sampling unit in the target population receives its appropriate chance of selection. The selection probabilities do not have to be equal but they should all be greater than zero. In fact, some designs bring gains in efficiency by not assigning equal probabilities to all of the distinct sampling units.

For a procedure that satisfies these bulleted properties it is possible to develop a mathematical theory for various methods of estimation based on probability sampling and to study the features of the estimation method (i.e., bias, precision, cost) although the details of the theory may be complex. If a particular probability sample design is properly executed, i.e., defining the universe, the frame, the sampling units, using proper randomization, accurately measuring the variables of interest, and using the correct formulas for estimation, then assertions that the sample and its resulting estimates are “not statistically valid” cannot legitimately be made. In other words, a probability sample and its results are always “valid.” Because of differences in the choice of a design, the level of available resources, and the method of estimation, however, some procedures lead to higher precision (smaller confidence intervals) than other methods. A feature of probability sampling is that the level of uncertainty can be incorporated into the estimate of overpayment as is discussed below. 8.4.3 - Selection of Period to be Reviewed and Composition of Universe (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) 8.4.3.1 - Selection of Period for Review (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Following selection of the provider or supplier, determine the time period and the number of days, weeks, months, or years, for which sampling units will be reviewed. The target universe shall be defined according to this period. The period of review is determined by considering several factors, including (but not limited to):

• How long the pattern of sustained or high level of payment error is believed to have existed;

• The volume of claims that are involved; • The length of time that a national coverage decision or regional or local

coverage policy has been in effect (i.e., should the provider or supplier have succeeded in adjusting their billing/utilization practices by now);

• The extent of prepayment review already conducted or currently being

conducted; • The dollar value of the claims that are involved relative to the cost effectiveness

of the sample; and/or,

• The applicable time periods for reopening claims (see the Medicare Claims Processing Manual, chapter 34 §10.6

NOTE: When sampling claims that are paid through cost report (as opposed to claims

paid under a PPS reimbursement methodology), all claims reviewed must be drawn from within a provider’s defined cost reporting year. If the period under review is greater than one year, select a separate sample for each cost-reporting year.

8.4.3.2 - Defining the Universe, the Sampling Unit, and the Sampling Frame (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The universe and sampling frame will usually cover all relevant claims or line items for the period under review. The discussion that follows assumes that the sampling unit is the claim, although this is not required. The sampling unit may also be a cluster of claims, as, for example, the patient, a treatment “day”, or any other sampling unit appropriate for the issue under review. 8.4.3.2.1 - Composition of the Universe (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) A. Part A Claims: For providers reimbursed through cost report, the universe of claims from which the sample is selected shall consist of fully and partially adjudicated claims obtained from the shared systems. For such claims, use the service date to match findings to the cost report. For providers reimbursed under PPS, the universe of claims from which the sample is selected will consist of all fully and partially paid claims submitted by the provider for the period under review. B. Part B Claims: The universe shall consist of all fully and partially paid claims submitted by the supplier for the period selected for review and for the sampling units to be reviewed. For example, if the review is of Physician X for the period January 1, 2002 through March 31, 2002, and laboratory and other diagnostic tests have been selected for review, the universe would include all fully and partially paid claims for laboratory and diagnostic tests billed by that physician for the selected time period. For some reviews, the period of review may best be defined in terms of the date(s) of service because changes in coverage policy may have occurred. 8.4.3.2.2 - The Sampling Unit (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Sampling units are the elements that are selected according to the design of the survey and the chosen method of statistical sampling. They may be an individual line(s) within claims, individual claims, or clusters of claims (e.g., a beneficiary). For example,

possible sampling units may include specific beneficiaries seen by a physician during the time period under review; or, claims for a specific item or service. In certain circumstances, e.g., multi-stage sample designs, other types of clusters of payments may be used. In principle, any type of sampling unit is permissible as long as the total aggregate of such units covers the population of potential mis-paid amounts. Unlike procedures for suppliers, overpayment projection and recovery procedures for providers and non-physician practitioners who bill intermediaries, in a non-PPS environment, must be designed so that overpayment amounts can be accurately reflected on the provider’s cost report. Therefore, sampling units must coincide with a projection methodology designed specifically for that type of provider to ensure that the results can be placed at the appropriate points on the provider’s cost report. The sample may be either claim-based or composed of specific line items. For example, home health cost reports are determined in units of “visits” for disciplines 1 through 6 and “lower of costs or charges” for drugs, supplies, etc. If claims are paid under cost report, the services reviewed and how those units link to the provider’s cost report must be known. Follow the instructions contained in section 3.10, but use the projection methodologies provided in PIM, Exhibits 9 through 12, for the appropriate provider type. PIM, Exhibits 9 through 12, are to be used only for claims not paid under PPS. 8.4.3.2.3 - The Sampling Frame (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The sampling frame is the “listing” of all the possible sampling units from which the sample is selected. The frame may be, for example, a list of all beneficiaries receiving items from a selected supplier, a list of all claims for which fully or partially favorable determinations have been issued, or a list of all the line items for specific items or services for which fully or partially favorable determinations have been issued. The ideal frame is a list that covers the target universe completely. In some cases the frame must be constructed by combining lists from several sources and duplication of sampling units may result. Although duplicate listings can be handled in various ways that do not invalidate the sample, it is recommended that duplicates be eliminated before selecting the sample. 8.4.4 - Sample Selection (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) 8.4.4.1 - Sample Design (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Identify the sample design to be followed. The most common designs used are simple random sampling, systematic sampling, stratified sampling, and cluster sampling, or a combination of these.

8.4.4.1.1 - Simple Random Sampling (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Simple random sampling involves using a random selection method to draw a fixed number of sampling units from the frame without replacement, i.e., not allowing the same sampling unit to be selected more than once. The random selection method must ensure that, given the desired sample size, each distinguishable set of sampling units has the same probability of selection as any other set - thus the method is a case of “equal probability sampling.” An example of simple random sampling is that of shuffling a deck of playing cards and dealing out a certain number of cards (although for such a design to qualify as probability sampling a randomization method that is more precise than hand shuffling and dealing would be required.) 8.4.4.1.2 - Systematic Sampling (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Systematic sampling requires that the frame of sampling units be numbered, in order, starting with the number one (1) and ending with a number equal to the size of the frame. Using a random start, the first sampling unit is selected according to that random number, and the remaining sampling units that comprise the sample are selected using a fixed interval thereafter. For example, if a systematic sample with size one-tenth of the frame size is desired, select a random number between one and ten, say that it is “6”, and then select every tenth unit thereafter, i.e., “16, 26, 36, …etc.” until the maximum unit number in the frame has been exceeded. 8.4.4.1.3 - Stratified Sampling (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Stratified sampling involves classifying the sampling units in the frame into non-overlapping groups, or strata. The stratification scheme should try to ensure that a sampling unit from a particular stratum is more likely to be similar in overpayment amount to others in its stratum than to sampling units in other strata. Although the amount of an overpayment cannot be known prior to review, it may be possible to stratify on an observable variable that is correlated with the overpayment amount of the sampling unit. Given a sample in which the total frame is covered by non-overlapping strata, if independent probability samples are selected from each of the strata, the design is called stratified sampling. The independent random samples from the strata need not have the same selection rates. A common situation is one in which the overpayment amount in a frame of claims is thought to be significantly correlated with the amount of the original payment to the provider or supplier. The frame may then be stratified into a number of distinct groups by the level of the original payment and separate simple random samples are drawn from each stratum. Separate estimates of overpayment are made for each stratum and the results combined to yield an overall projected overpayment. The main object of stratification is to define the strata in a way that will reduce the margin of error in the estimate below that which would be attained by other sampling

methods, as well as to obtain an unbiased estimate or an estimate with an acceptable bias. The standard literature, including that referenced in Section 3.10.10, contains a number of different plans; the suitability of a particular method of stratification depends on the particular problem being reviewed, and the resources allotted to reviewing the problem. Additional discussion of stratified sampling is provided in Section 8.4.11.1. 8.4.4.1.4 - Cluster Sampling (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Cluster sampling involves drawing a random sample of clusters and reviewing either all units or a sample of units selected from each of the sampled clusters. Unlike strata, clusters are groups of units that do not necessarily have strong similarities, but for which their selection and review as clusters is more efficient economically than, for example, simple random sampling. For example, if the sampling unit is a beneficiary and the plan is to review each of the set of payments for each selected beneficiary, then the design is an example of cluster sampling with each beneficiary constituting a cluster of payments. The main point to remember (when sampling all the units in the cluster) is that the sample size for purposes of estimating the sampling error of the estimate is the number of clusters, not the total number of individual payments that are reviewed. A challenge to the validity of a cluster sample that is sometimes made is that the number of sampling units in a cluster is too small. (A similar challenge to stratified sampling is also raised – i.e., that the number of sampling units in a stratum is too small). Such a challenge is usually misguided since the estimate of the total overpayment is a combination of the individual cluster (or, in the case of stratified sampling, stratum) estimates; therefore the overall sample size is important, but the individual cluster (or stratum) sample sizes are usually not critical. Additional discussion of cluster sampling is provided in Section 8.4.11.2. Both stratification and cluster sampling involve the grouping of more elementary units. The former is frequently recommended when there is sufficient prior knowledge to group units that are similar in some aspect and potentially different from other units. The latter is frequently recommended when there are natural groupings that make a study more cost effective. When carried out according to the rules of probability sampling both of the methods, or a combination, are valid. The use of any of the methods described in this section will produce valid results when done properly. 8.4.4.1.5 - Design Combinations (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) A sample design may combine two or more of the methods discussed above. For example, clusters may be stratified before selection; systematic selection rather than simple random sampling may be used for selecting units within strata; or clusters may be subsampled using either simple random sampling or systematic sampling, to cite some of the possible combinations of techniques.

The benefits of stratification by claim amount may be achieved without actually stratifying if the frame is arranged in ascending order by the original payment amount and systematic sampling applied with a random start. That is because the systematic selection “balances out” the sample over the different levels of original payment in a manner similar to the effect of formal stratification. Thus systematic selection is often used in the hope that it will result in increased precision through “implicit stratification.” 8.4.4.2 - Random Number Selection (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The PSC or ZPIC BI unit or the contractor MR unit shall identify the source of the random numbers used to select the individual sampling units. The PSC or ZPIC BI unit or the contractor MR unit shall also document the program and its algorithm or table that is used; this documentation becomes part of the record of the sampling and must be available for review. The PSC or ZPIC BI unit or the contractor MR unit shall document any starting point if using a random number table or drawing a systematic sample. In addition, the PSC or ZPIC BI units or the contractor MR units shall document the known seed value if a computer algorithm is used. The PSC or ZPIC BI units or the contractor MR units shall document all steps taken in the random selection process exactly as done to ensure that the necessary information is available for anyone attempting to replicate the sample selection. There are a number of well-known, reputable software statistical packages (SPSS, SAS, etc.) and tables that may be used for generating a sample. One such package is RAT-STATS, available (at time of release of these instructions) through the Department of Health and Human Services, Office of Inspector General Web Site. It is emphasized that the different packages offer a variety of programs for sample generation and do not all contain the same program features or the same ease in operation. For any particular problem, the PSC or ZPIC BI unit’s or the contractor MR unit’s statistician or systems programmer shall determine which package is best suited to the problem being reviewed. 8.4.4.3 - Determining Sample Size (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The size of the sample (i.e., the number of sampling units) will have a direct bearing on the precision of the estimated overpayment, but it is not the only factor that influences precision. The standard error of the estimator also depends on (1) the underlying variation in the target population, (2) the particular sampling method that is employed (such as simple random, stratified, or cluster sampling), and (3) the particular form of the estimator that is used (e.g., simple expansion of the sample total by dividing by the selection rate, or more complicated methods such as ratio estimation). It is neither possible nor desirable to specify a minimum sample size that applies to all situations. A determination of sample size may take into account many things, including the method of sample selection, the estimator of overpayment, and prior knowledge (based on experience) of the variability of the possible overpayments that may be contained in the total population of sampling units.

In addition to the above considerations, real-world economic constraints shall be taken into account. As stated earlier, sampling is used when it is not administratively feasible to review every sampling unit in the target population. In determining the sample size to be used, the PSC or ZPIC BI unit or the contractor MR unit shall also consider their available resources. That does not mean, however, that the resulting estimate of overpayment is not valid, so long as proper procedures for the execution of probability sampling have been followed. A challenge to the validity of the sample that is sometimes made is that the particular sample size is too small to yield meaningful results. Such a challenge is without merit as it fails to take into account all of the other factors that are involved in the sample design. 8.4.4.4 - Documentation of Sampling Methodology (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The PSC or ZPIC BI unit or the contractor MR unit shall maintain complete documentation of the sampling methodology that was followed. 8.4.4.4.1 - Documentation of Universe and Frame (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) An explicit statement of how the universe is defined and elements included shall be made and maintained in writing. Further, the form of the frame and specific details as to the period covered, definition of the sampling unit(s), identifiers for the sampling units (e.g., claim numbers, carrier control numbers), and dates of service and source shall be specified and recorded in your record of how the sampling was done. A record shall be kept of the random numbers actually used in the sample and how they were selected. Sufficient documentation shall be kept so that the sampling frame can be re-created, should the methodology be challenged. The PSC or ZPIC BI units or the contractor MR units shall keep a copy of the frame. 8.4.4.4.2 - Arrangement and Control Totals (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) It is often convenient in frame preparation to array the universe elements by payment amount, e.g., low to high values, especially when stratification is used. At the same time, tabulate control totals for the numbers of elements and payment amounts. 8.4.4.4.3 - Worksheets (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The PSC or ZPIC BI units or the contractor MR units shall maintain documentation of the review and sampling process. All worksheets used by reviewers shall contain sufficient information that allows for identification of the claim or item reviewed. Such information may include, for example:

• Name and identification number of the provider or supplier; • Name and title of reviewer; • The Health Insurance Claim Number (HICN), the unique claim identifier (e.g., the

claim control number), and the line item identifier; • Identification of each sampling unit and its components (e.g., UB-92 or attached

medical information) • Stratum and cluster identifiers, if applicable; • The amount of the original submitted charges (in column format); • Any other information required by the cost report worksheets in PIM Exhibits 9

through 12; • The amount paid; • The amount that should have been paid (either over or underpaid amount); and, • The date(s) of service. 8.4.4.4.4 - Overpayment/Underpayment Worksheets (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Worksheets shall be used in calculating the net overpayment. The worksheet shall include data on the claim number, line item, amount paid, audited value, amount overpaid, reason for disallowance, etc., so that each step in the overpayment calculation is clearly shown. Underpayments identified during reviews shall be similarly documented. 8.4.4.5 - Informational Copies to Primary GTL, Associate GTL, SME or CMS RO (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The PSC or ZPIC BI units or the contractor MR units shall send informational copies of the statistician-approved sampling methodology to their Primary GTL, Associate GTL, SME or CMS RO. The Primary GTL, Associate GTL, SME or CMS RO will keep the methodology on file and will forward to CO upon request. If this sampling methodology is applied routinely and repeatedly, the PSC or ZPIC BI units or the contractor MR units shall not repeatedly send the methodology to the Primary GTL, Associate GTL, SME or CMS RO.

8.4.5 - Calculating the Estimated Overpayment (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) 8.4.5.1 - The Point Estimate (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) In simple random or systematic sampling the total overpayment in the frame may be estimated by calculating the mean overpayment, net of underpayment, in the sample and multiplying it by the number of units in the frame. In this estimation procedure, which is unbiased, the amount of overpayment dollars in the sample is expanded to yield an overpayment figure for the universe. The method is equivalent to dividing the total sample overpayment by the selection rate. The resulting estimated total is called the point estimate of the overpayment, i.e., the difference between what was paid and what should have been paid. In stratified sampling, an estimate is found for each stratum separately, and the weighted stratum estimates are added together to produce an overall point estimate. In most situations the lower limit of a one-sided 90 percent confidence interval shall be used as the amount of overpayment to be demanded for recovery from the provider or supplier. The details of the calculation of this lower limit involve subtracting some multiple of the estimated standard error from the point estimate, thus yielding a lower figure. This procedure, which, through confidence interval estimation, incorporates the uncertainty inherent in the sample design, is a conservative method that works to the financial advantage of the provider or supplier. That is, it yields a demand amount for recovery that is very likely less than the true amount of overpayment, and it allows a reasonable recovery without requiring the tight precision that might be needed to support a demand for the point estimate. However, the PSC or ZPIC BI unit or the contractor MR unit is not precluded from demanding the point estimate where high precision has been achieved. Other methods of obtaining the point estimate are discussed in the standard textbooks on sampling theory. Alternatives to the simple expansion method that make use of auxiliary variables include ratio and regression estimation. Under the appropriate conditions, ratio or regression methods can result in smaller margins of error than the simple expansion method. For example, if, as discussed earlier, it is believed that the overpayment for a sample unit is strongly correlated with the original paid amount, the ratio estimator may be efficient. The ratio estimator is the ratio of the sample net overpayment to the sample total original payment multiplied by the total of original paid dollars in the frame. If the actual correlation between the overpayment and the original paid amount is high enough, greater precision in estimation will be attained, i.e., the lower limit of the one-sided 90 percent confidence interval will be closer to the point estimate. Exercise caution about using alternatives such as ratio or regression estimation because serious biases can be introduced if sample sizes are very small. (The term bias is used here in a technical sense and does not imply a finding that treats the provider or supplier unfairly. A biased estimator is often used rather than an unbiased estimator because the advantage of its greater precision outweighs the tendency of the point estimate to be a bit high or low.)

8.4.5.2 - Calculation of the Estimated Overpayment Amount (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The results of the sampling unit reviews are used to project an estimate of the overpayment amount. Each result shall be recorded except that a sampling unit’s overpayment shall be set to zero if there is a limitation on liability determination made to waive provider or supplier liability for that sampling unit (per provisions found in §1879 of the Social Security Act (the Act)) and/or there is a determination that the provider or supplier is without fault as to that sampling unit overpayment (per provisions found in §1870 of the Act). Sampling units for which the requested records were not provided are to be treated as improper payments (i.e., as overpayments). Sampling units that are found to be underpayments, in whole or in part, are recorded as negative overpayments and shall also be used in calculating the estimated overpayment. 8.4.6 - Actions to be Performed Following Selection of Provider or Supplier and Sample (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) NOTE: The instructions in this section dealing with notification and determination of location of the review do not supersede instructions for PSC or ZPIC BI units or the contractor MR units that are using statistical sampling for overpayment estimation as part of an investigation, either planned or on-going, into potential Medicare fraud. 8.4.6.1 – Notification of Provider or Supplier of the Review and Selection of the Review Site (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) The PSC or ZPIC BI unit or the contractor MR unit shall first determine whether it will be giving advance notification to the provider or supplier of the review. Although in most cases the PSC or ZPIC BI unit or the contractor MR unit shall give prior notification, the provider or supplier is not always notified before the start of the review. When not giving advance notice, the PSC or ZPIC BI unit or PSC MR unit shall obtain the advance approval of the Primary GTL; and the contractor MR unit shall obtain the advance approval of the CMS RO. When giving advance notice, provide written notification by certified mail with return receipt requested (retain all receipts). Second, regardless of whether you give advance notice or not, you shall determine where to conduct the review of the medical and other records: either at the provider or supplier’s site(s) or at your office (PSC or ZPIC BI units or contractor MR units). 8.4.6.1.1 - Written Notification of Review (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) You shall include at least the following in the notification of review:

• an explanation of why the review is being conducted (i.e., why the provider or

supplier was selected),

• the time period under review,

• a list of claims that require medical records or other supporting documentation,

• a statement of where the review will take place (provider/supplier office or contractor site),

• information on appeal rights,

• an explanation of how results will be projected to the universe if claims are denied upon review and an overpayment is determined to exist, and

• an explanation of the possible methods of monetary recovery if an overpayment is determined to exist. When advance notification is given, providers and suppliers have 30 calendar days to submit (for PSC or ZPIC BI unit or contractor MR unit site reviews) or make available (for provider/supplier site reviews) the requested documentation. Advise the provider or supplier that for requested documentation that is not submitted or made available by the end of 30 calendar days, you will start the review and you will deny those claims for which there is no documentation. The time limit for submission or production of requested documentation may be extended at your discretion. NOTE: You do not have to request all documentation at the time of notification of review. For example, you may decide to request one-half of the documentation before you arrive, and then request the other half following your arrival at the provider/supplier’s site. When advance notification is not given, you shall give the provider or supplier the written notification of review when you arrive at their site. 8.4.6.1.2 - Determining Review Site (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) A. Provider/Supplier Site Reviews Provider/supplier site reviews are performed at the provider’s or supplier’s location(s). Considerations in determining whether to conduct the review at the office of the provider or supplier include, but are not limited to, the following:

• the extent of aberrant billing or utilization patterns that have been identified;

• the presence of multiple program integrity issues;

• evidence or likelihood of fraud or abuse; and/or,

• past failure(s) of the provider or supplier to submit requested medical records in a timely manner or as requested. B. PSC or ZPIC BI Unit or Contractor MR Unit Site Reviews The PSC or ZPIC BI unit or the contractor MR unit site reviews are performed at a location of the PSC or ZPIC BI unit or the contractor MR unit. 8.4.6.2 - Meetings to Start and End the Review (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) In-person meetings to start and end the review are encouraged, but are not required or always feasible. If you hold an in-person meeting at the start of the review, explain both the scope and purpose of the review as well as discuss what will happen once you have completed the review. Attempt to answer all questions of the provider or supplier related to the review. During an exit meeting, you may discuss the basic or preliminary findings of the review. Give the provider or supplier an opportunity to discuss or comment on the claims decisions that were made. Advise the provider or supplier that a demand letter detailing the results of the review and the statistical sampling will be sent if an overpayment is determined to exist. 8.4.6.3 - Conducting the Review (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Following your receipt of the requested documentation (or the end of the period to submit or make available the requested documentation, whichever comes first), start your review of the claims. You may ask for additional documentation as necessary for an objective and thorough evaluation of the payments that have been made, but you do not have to hold up conducting the review if the documents are not provided within a reasonable time frame. Use physician consultants and other health professionals in the various specialties as necessary to review or approve decisions involving medical judgment. The review decision is made on the basis of the Medicare law, HCFA/CMS rulings, regulations, national coverage determinations, Medicare instructions, and regional/local contractor medical review policies that were in effect at the time the item(s) or service(s) was provided. Document all findings made so that it is apparent from your written documentation if the initial determination has been reversed. Document the amount of all overpayments and underpayments and how they were determined.

You are encouraged to complete your review and calculate the net overpayment within 90 calendar days of the start of the review (i.e., within 90 calendar days after you have either received the requested documentation or the time to submit or make available the records has passed, whichever comes first). However, there may be extenuating circumstances or circumstances out of your control where you may not be able to complete the review within this time period (e.g., you have made a fraud referral to the OIG and are awaiting their response before pursuing an overpayment). Your documentation of overpayment and underpayment determinations shall be clear and concise. Include copies of the local medical review policy and any applicable references needed to support individual case determinations. Compliance with these requirements facilitates adherence to the provider and supplier notification requirements. 8.4.7 - Overpayment Recovery (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) 8.4.7.1 - Recovery From Provider or Supplier (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Once an overpayment has been determined to exist, proceed with recovery based on applicable instructions. (See Publication 100-6, Financial Management Manual, chapter 3.) Include in the overpayment demand letter information about the review and statistical sampling methodology that was followed. For PSCs and ZPICs, only ACs or MACs shall issue demand letters and recoup the overpayment. The explanation of the sampling methodology that was followed shall include:

• a description of the universe, the frame, and the sample design; • a definition of the sampling unit,

• the sample selection procedure followed, and the numbers and definitions of the

strata and size of the sample, including allocations, if stratified;

• the time period under review;

• the sample results, including the overpayment estimation methodology and the calculated sampling error as estimated from the sample results; and

• the amount of the actual overpayment/underpayment from each of the claims reviewed. Also include a list of any problems/issued identified during the review, and any recommended corrective actions.

8.4.7.2 - Informational Copy to Primary GTL, Associate GTL, SME or CMS RO (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Send an informational copy of the demand letter to the Primary GTL, Associate GTL, SME or CMS RO. They will maintain copies of demand letters and will forward to CO upon request. If the demand letter is used routinely and repeatedly, you shall not repeatedly send it to the Primary GTL, Associate GTL, SME or CMS RO. 8.4.8 - Corrective Actions (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Take or recommend other corrective actions you deem necessary (such as payment suspension, imposition of civil money penalties, institution of pre- or post-payment review, additional edits, etc.) based upon your findings during or after the review. 8.4.9 - Changes Resulting From Appeals (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) If the decision issued on appeal contains either a finding that the sampling methodology was not valid, and/or reverses the revised initial claim determination, you shall take appropriate action to adjust the extrapolation of overpayment. 8.4.9.1 - Sampling Methodology Overturned (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) If the decision issued on appeal contains a finding that the sampling methodology was not valid, there are several options for revising the estimated overpayment based upon the appellate decision: A. If the decision issued on appeal permits correction of errors in the sampling methodology, you shall revise the overpayment determination after making the corrections. Consult with your Primary GTL, Associate GTL, SME or CMS RO to confirm that this course of action is consistent with the decision of the hearing officer (HO), administrative law judge (ALJ) or Departmental Appeals Board (DAB), or with the court order. B. You may elect to recover the actual overpayments related to the sampled claims and then initiate a new review of the provider or supplier. If the actual overpayments related to the sampling units in the original review have been recovered, then these individual sampling units shall be eliminated from the sampling frame used for any new review. Consult with your Primary GTL, Associate GTL, SME or CMS RO to confirm that this course of action is consistent with the decision of the HO, ALJ or DAB, or with the court order.

C. You may conduct a new review (using a new, valid methodology) for the same time period as was covered by the previous review. If this option is chosen, you shall not recover the actual overpayments on any of the sample claims found to be in error in the original sample. Before employing this option, consult with your Primary GTL, Associate GTL, SME or CMS RO to verify that this course of action is consistent with the decision of the HO, ALJ or DAB, or with the court order. 8.4.9.2 - Revised Initial Determination (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) If the decision on appeal upholds the sampling methodology but reverses one or more of the revised initial claim determinations, the estimate of overpayment shall be recomputed and a revised projection of overpayment issued. 8.4.10 - Resources (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) American Institute of Certified Public Accountants, Statistical Sampling Subcommittee, Audit Sampling, 1999. Arkin, H., Handbook of Sampling for Auditing and Accounting, 1984. Cochran, W. G., Sampling Techniques, 3rd ed., New York: John Wiley and Sons, 1977. Deming, W. E., Sample Design in Business Research, New York: John Wiley and Sons, 1960 (Paperback 1990). Hansen, M. H., Hurwitz, W. W., and Madow, W. G., Sample Survey Methods and Theory, New York: John Wiley and Sons, 1953 (Paperback 1993). Hedayat, A., Bekas, K. S., Design and Inference in Finite Population Sampling, John Wiley & Sons, New York, 1991. Kish, L., Survey Sampling, New York: John Wiley and Sons, 1967, 2nd printing. (Paperback 1995). Levy, P. and Lemeshow, S., Sampling of Populations Methods and Applications, 3rd ed., John Wiley & Sons, 1999. Scheaffer, R. L., Mendenhall, W., and Ott, L., Elementary Survey Sampling, 5th ed., Duxbury Press, 1996. Som, R. K., Practical Sampling Techniques, M. Dekker, New York, 1996, 2 nd ed.

8.4.11 - Additional Discussion of Stratified Sampling and Cluster Sampling (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) 8.4.11.1 – Stratified Sampling (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Generally, one defines strata to make them as internally homogeneous as possible with respect to overpayment amounts, which is equivalent to making the mean overpayments for different strata as different as possible. Typically, a proportionately stratified design with a given total sample size will yield an estimate that is more precise than a simple random sample of the same size without stratifying. The one highly unusual exception is one where the variability from stratum mean to stratum mean is small relative to the average variability within each stratum. In this case, the precision would likely be reduced, but the result would be valid. It is extremely unlikely, however, that such a situation would ever occur in practice. Stratifying on a variable that is a reasonable surrogate for an overpayment can do no harm, and may greatly improve the precision of the estimated overpayment over simple random sampling. While it is a good idea to stratify whenever there is a reasonable basis for grouping the sampling units, failure to stratify does not invalidate the sample, nor does it bias the results. If it is believed that the amount of overpayment is correlated with the amount of the original payment and the universe distribution of paid amounts is skewed to the right, i.e., with a set of extremely high values, it may be advantageous to define a “certainty stratum”, selecting all of the sampling units starting with the largest value and working backward to the left of the distribution. When a stratum is sampled with certainty, i.e., auditing all of the sample units contained therein, the contribution of that stratum to the overall sampling error is zero. In that manner, extremely large overpayments in the sample are prevented from causing poor precision in estimation. In practice, the decision of whether or not to sample the right tail with certainty depends on fairly accurate prior knowledge of the distribution of overpayments, and also on the ability to totally audit one stratum while having sufficient resources left over to sample from each of the remaining strata. Stratification works best if one has sufficient information on particular subgroups in the population to form reasonable strata. In addition to improving precision there are a number of reasons to stratify, e.g., ensuring that particular types of claims, line items or coding types are sampled, gaining information about overpayments for a particular type of service as well as an overall estimate, and assuring that certain rarely occurring types of services are represented. Not all stratifications will improve precision, but such stratifications may be advantageous and are valid. Given the definition of a set of strata, the designer of the sample must decide how to allocate a sample of a certain total size to the individual strata. In other words, how much of the sample should be selected from Stratum 1, how much from Stratum 2, etc.? As shown in the standard textbooks, there is a method of “optimal allocation,” i.e., one

designed to maximize the precision of the estimated potential overpayment, assuming that one has a good idea of the values of the variances within each of the strata. Absent that kind of prior knowledge, however, a safe approach is to allocate proportionately. That is, the total sample is divided up into individual stratum samples so that, as nearly as possible, the stratum sample sizes are in a fixed proportion to the sizes of the individual stratum frames. It is emphasized, however, that even if the allocation is not optimal, using stratification with simple random sampling within each stratum does not introduce bias, and in almost all circumstances proportionate allocation will reduce the sampling error over that for an unstratified simple random sample. 8.4.11.2 - Cluster Sampling (Rev. 377, Issued: 05-27-11, Effective: 06-28-11, Implementation: 06-28-11) Selecting payments in clusters rather than individually usually leads to a reduction in the precision of estimation. However, your reasons for using cluster sampling instead of simple random sampling may be driven by necessity and/or cost-savings related to the location of records or the nature of a record. For example, for medical review to determine the appropriateness of certain charges for a beneficiary it may be necessary to examine the complete medical record of the patient. This then may allow for review of claims for several services falling within the selected review period. In another instance, the medical records that you must review may be physically located in a cluster (e.g., the same warehouse, the same file drawer, the same folder) with the medical records for other similar claims and it is cost effective to select units from the same location. Whenever the cost in time and other resources of selecting and auditing clusters is the same as the cost of simple random sampling the same number of payments, it is better to use simple random sampling because greater precision will be attained. When reviewing all the units in each cluster, the sample size is the number of clusters, not the number of units reviewed. This is single-stage cluster sampling, a method frequently used when sampling beneficiaries. One may choose to review a sample of units within each cluster rather than all units. Textbooks that cover the topic of multi-stage sampling provide formulas for estimating the precision of such sample designs. One example for which multi-stage sampling might be an appropriate choice of design is the case of reviewing a supplier chain where records are spread out among many locations. The first-stage selection would be a sample of locations. At the second stage a subsample of records would be selected from each sampled location.

Transmittals Issued for this Chapter

Rev # Issue Date Subject Impl Date CR# R377PI 05/27/2011 Program Integrity Manual Reorganization of

Chapters 3 and 8 06/28/2011 6560

R71PI 04/09/2004 Rewrite of Program Integrity Manual (except Chapter 10) to Apply to PSCs

05/10/2004 3030

R03PIM 11/22/2000 Complete Replacement of PIM Revision 1. NA 1292 R01PIM 06/2000 Initial Release of Manual NA 931

Back to top of chapter

http://www.cms.hhs.gov/transmittals/downloads/R377PI.pdf�

http://www.cms.hhs.gov/Transmittals/Downloads/R71PI.pdf�

http://www.cms.hhs.gov/Transmittals/Downloads/R3PIM.pdf�

http://www.cms.hhs.gov/Transmittals/Downloads/R1PIM.pdf�

Version 2

RAT-STATS Companion Manual

TABLE OF CONTENTS PageRandom Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1

Single-Stage Random Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2

Sets of Two Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3

Sets of Three Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4

Sets of Four Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5

Frames - Single Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-6

Frames - Sets of Two . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-8

RHC Sample Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-11

Procedure for Two-Stage RHC Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-11

Procedure for Three-Stage RHC Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-18

Summary of Input for RHC Sample Selection . . . . . . . . . . . . . . . . . . . . . . . . 1-20

Generating Spares for RHC Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-24

Comparison of RHC and Multistage SRS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-29

Attribute Appraisals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1

Unrestricted Attribute Appraisal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2

Stratified Attribute Appraisal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-6

Two-Stage Unrestricted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-11

Three-Stage Unrestricted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-19

RHC Two Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-28

RHC Three Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-36

Stratified Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-62

Stratified Multistage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-68

Variable Appraisals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1

Unrestricted Variable Appraisal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2

Stratified Variable Appraisal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8

Using a Stratified Sample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8

Strata Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-11

Two-Stage Unrestricted Variable Appraisal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-17

Three-Stage Unrestricted Variable Appraisal . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-24

RHC Two Stage Variable Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-35

RHC Three Stage Variable Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-48

Stratified Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-77

Stratified Multistage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-82

Post Stratification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-91

Unknown Universe Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-99

Sample Size Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-1

Variable Sample Size Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2

Unrestricted Using a Probe Sample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2

Unrestricted Using Estimated Error Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-7

Stratified . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-11

Total Sample Size Unknown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-11

Total Sample Size Known . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-16

Attribute Sample Size Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-22

PREFACE

The purpose of this manual is to provide:

# an overview of each program in the Windows version of RAT-STATS,

# examples illustrating the application of the software,

# snapshots of data sets used by the programs,

# some discussion regarding the program output, and

# formulas used within the software.

The intent is for the auditor/specialist to use as much of this discussion as he/she finds helpful.

While the RAT-STATS Users Guide gives descriptions of program input and output, this

Companion Manual should provide insight as to how to better use the software and exactly how

the program derives the results. The formulas are provided so that OAS has a single source for

all formulas in the event that a question is raised as to exactly how a particular result was

obtained.

We hope you find that the manual makes the OAS software easier to understand and easier to

apply. Please pass on any suggestions or corrections to Office of Inspector General, Office of

Public Affairs at [email protected].

RAT-STATS Companion Manual Random Numbers

(Rev. 10/2004) Page 1-1

# Single-Stage Random Numbers

# Sets of Two Numbers

# Sets of Three Numbers

# Sets of Four Numbers

# Frames - Single Stage

# Frames - Sets of Two

# RHC Sample Selection

RANDOM NUMBERS

Whatever statistical sampling design you end up using (including stratified and/or multistage), at

some point in the data collection you will need one or more random samples. The next section,

dealing with Unrestricted Random Sampling, will examine the mechanics and estimation

procedures using such a sample in detail, but first it is necessary to discuss procedures for

generating a random sample. A number of programs exist for such purposes; namely:

Single-Stage Random Numbers RAT-STATS Companion Manual

Page 1-2 (Rev. 10/2004)

Single-Stage Random Numbers

This program generates an unduplicated quantity of random numbers. Random values (the

output from this program) can be output in sequential order, random order, or a mixture of both.

For values in sequential order, you will see the values printed in sequential order, beginning with

the smallest selected item number and proceeding to the largest item number. For values in

random order, the items are printed in the order in which they were selected by the program.

Example 1. A universe contains 1,000 payments and a simple random sample of 10 payments

(with four spares) is needed. What items should be selected?

Solution: Using this program and a seed value of 12345, the sampled payments are those

numbered as follows:

9, 236, 337, 340, 346, 497, 556, 641, 658, and 884

The four spares are payments 404, 624, 927, and 947.

RAT-STATS Companion Manual Sets of Two Numbers

(Rev. 10/2004) Page 1-3

Sets of Two Numbers

This program will generate unduplicated pairs of random numbers. This is useful when sample

items are selected through a two-step process (e.g., page number and line number).

Random values (the output from this program) can be output in sequential order, random order,

or a mixture of both. Values in sequential order will be printed in sequential order, beginning

with the smallest selected item number and proceeding to the largest item number. For values in


Example 2. Items are selected from a computer printout that had pages numbered 1 through 658

and had 66 lines on each page. A simple random sample of 10 items (with four spares) is

needed. Which items should be selected?

Solution: Using this program and a seed value of 12345, the sampled payments are:

PAGE: 224 258 266 327 366 400 422 433 561 610

ITEM: 23 37 42 1 16 7 23 59 40 63

The spares are: PAGE: 579 109 188 330

ITEM: 61 54 50 49

Sets of Three Numbers RAT-STATS Companion Manual

Page 1-4 (Rev. 10/2004)

Sets of Three Numbers

This program will generate unduplicated sets of three random numbers. This should be used

when sample items are selected through a three-step process (e.g., month, page, and line

number).


or a mixture of both. Values in sequential order will be printed in sequential order, beginning

with the smallest selected item number and proceeding to the largest item number. For values in


Example 3. Same as Example 2, where the pages are numbered 1 through 658 for each month.

Here the universe consists of 1 year's worth (12 months) of computer printouts. We need four

sample items and two spares.

Solution: Using this program and a seed value of 12345, the sampled items are:

MONTH: 3 5 6 8

PAGE: 224 266 6 582

ITEM: 23 42 37 43

The two spares are: MONTH: 12 8

PAGE: 623 258

ITEM: 57 37

RAT-STATS Companion Manual Sets of Four Numbers

(Rev. 10/2004) Page 1-5

Sets of Four Numbers

This program will generate unduplicated sets of four random numbers. This should be used

when sample items are selected through a four-step process (e.g., year, month, page, and line

number).


or a mixture of both. For values in sequential order, you will see the values printed in sequential

order, beginning with the smallest selected item number and proceeding to the largest item

number. For values in random order, the items are printed in the order in which they were

selected by the program.

Example 4. Same as Example 3, where the pages are numbered 1 through 658 for each month

and year (total of 5 years). We need three sample items and two spares.

Solution: Using this program and a seed value of 12345, the sampled items are:

YEAR: 2 3 4

MONTH: 5 1 5

PAGE: 433 366 266

ITEM: 59 16 42

The two spares are: YEAR: 5 2

MONTH: 12 7

PAGE: 561 400

ITEM: 40 7

Frames - Single Stage RAT-STATS Companion Manual

Page 1-6 (Rev. 10/2004)

Frames - Single Stage

This program will generate an unduplicated set of random numbers which is useful when the

universe of sampling items either (1) contains gaps of numbers or (2) the numbering system

repeats within the universe. For instance, the universe of items consists of two frames,

numbered 1 through 1,050 and 8,405 through 9,565.



order, beginning with the smallest selected item number in the first frame (if any) and

proceeding to the largest item number in the last frame (if any). For values in random order, the

items are printed in the order in which they were selected by the program.

Example 5. A universe of items that refer to payment of a particular medical procedure are

numbered as follows:

1 - 1,050 (frame 1)

and 8,405 - 9,565 (frame 2)

A sample of five items is needed. Three of these items should be in sequential order and the

remaining two in random order.

Solution: Using this program and a seed number of 12345, the three sample items in sequential

order are:

FRAME ITEM NUMBER 1 20 2 8,452 2 8,584

RAT-STATS Companion Manual Frames - Single Stage

(Rev. 10/2004) Page 1-7

The two items in random order are

FRAME ITEM NUMBER 1 520 1 752

Explanation: For this example there are 2,211 items in the frame since there are 1,050 items in

the first frame and (9,565 - 8,405 + 1), i.e., 1,161 items, in the second frame. For the sequential

items, three values between 1 and 2,211 are generated. These values are 20, 1,098, and 1,230.

Since 1,098 is outside the first frame, it is in the second frame; in particular, its location in the

second frame would be

[8,405 + (1,098 - 1,050) - 1]

that is, item number 8,452. Similarly, the value of 1,230 points to item number 8,584.

Similarly, for the two items in random order, the program generated two values between 1 and

2,211. These values were 520 and 752. Since both values are less than 1,050, these locations

are items 520 and 752 in the first frame.

Frames - Sets of Two RAT-STATS Companion Manual

Page 1-8 (Rev. 10/2004)

Frames - Sets of Two

This program is a combination of two programs, Frames - Single Stage and Sets of Two

Numbers. The program will generate an unduplicated set of random numbers which should be

used when (1) pairs of numbers are used to locate sample items (as in Sets of Two Numbers) and

(2) the universe has gaps or the numbering system repeats (as in Frames - Single Stage).



order, beginning with the smallest selected item number in the first frame (if any) and

proceeding to the largest item number in the last frame (if any). For values in random order, the

items are printed in the order in which they were selected by the program.

Example 6. A universe of transactions consists of three sets of computer pages, numbered as

follows:

FRAME RANGE (Page Numbers) 1 1 - 100 2 1 - 456 3 45 - 832

In addition, within each frame there are an equal number of line items per page. The range

within each frame is:

FRAME RANGE (Number of Lines) 1 1 - 66 2 1 - 66 3 1 - 66

A sample of three items in sequential order and two items in random order is needed.

RAT-STATS Companion Manual Frames - Sets of Two

(Rev. 10/2004) Page 1-9

Solution: Using this program and a seed value of 12345, the three sample items in sequential

order are:

FRAME PAGE NO. LINE 1 12 45 3 156 21 3 236 14

The two items in random order are:

FRAME PAGE NO. LINE 2 216 64 2 357 36

Explanation: For this example, the frame consists of

(100)(66) + (456)(66) + (832 - 45 + 1)(66)

= 6,600 + 30,096 + 52,008 = 88,704 items.

Random numbers between 1 and 88,704 are generated. Values between 1 and 6,600 will be in

the first frame; values between 6,601 and 6,600 + 30,096 = 36,696 are in the second frame; and

values between 36,697 and 88,704 will come from the third frame.

The three random values (not the spares) generated are 771, 44,043, and 49,316.

To find the value corresponding to 771:

1. Find (771/66) + 1 = 12.682. The integer part of this is 12. So, this value is on page number

(subframe) 12 of frame 1.

2. The decimal part of this number is .682. Multiply this by 66 (the number of lines per page in

the first frame for this example) and round to the nearest integer. This is 45. This item is on line

45 of page 12.

Frames - Sets of Two RAT-STATS Companion Manual

Page 1-10 (Rev. 10/2004)

To find the sample value corresponding to 44,043: This value is larger than 36,696 so it is in the

third frame.

1. Find [(44,043 - 36,696)/66] + 1 = 156.318. Here, 45 is the low number (input) for frame 3.

The integer part of this value is 156. So, this value is on page number (subframe) 156 of

frame 2.

2. The decimal part of this number is .318. Multiply this by 66 (the number of lines per page in

the second frame for this example) and round to the nearest integer. This is 21. This item is

on line 21 of page 156.

Finally, consider the third randomly generated value of 49,316. This is 36,696, so it is in the

third frame.

1. Find [(49,316 - 36,696)/66] + 45 = 236.212. The integer part of 236.212 is 236. So, this

value is on page number 236 of frame 3.

2. The decimal part of this number is .212. Multiply this by 66 (the number of lines per page)

and round to the nearest integer. This is 14. This item is on line 14 of page 236.

RAT-STATS Companion Manual RHC Sample Selection

(Rev. 10/2004) Page 1-11

RHC Sample Selection

The RHC selection/appraisal procedure is named after three statisticians -- J.N.K. Rao, H.O.

Hartley, and William Cochran -- and was originally proposed in 1962. This procedure is

essentially the same as single-stage SRS sampling except that the size of each primary unit

(cluster) is used to select the sample. It provides a method of sample selection that allows

sampling without replacement (the usual procedure) while “maintaining the flavor” of using

probability proportional to size. It can be used to select primary units (P.U.s) in a two-stage

design or primary and secondary units (S.U.s) in a three-stage design.

Comment: Strictly speaking, you cannot use pure probability proportional to size (pps)

sampling when sampling without replacement. To understand why, consider a situation in which

a population contains 10 primary units with seven “large” P.U.s and three “small” ones. If the

sample size is eight, then one of the small P.U.s must be selected, regardless of its small size.

The RHC procedure is not pure pps sampling, but comes very close while allowing the auditor to

sample without replacement.

Procedure for Two-Stage RHC Sampling

Suppose that you have N P.U.s and you want a sample of n P.U.s. The procedure is to:

1. Randomly put (partition) the N P.U.s into n groups (no attention to size here).2. Within each of the n groups, select one P.U. using pps.

Example 7. N = 15, n = 3

1. Generate 3 groups, each containing 5 P.U.s

RHC Sample Selection RAT-STATS Companion Manual

Page 1-12 (Rev. 10/2004)

One possibility: Generate 15 random numbers between 0 and 1. Suppose the smallest value

is in location 8, next largest in location 5, next largest in location 13, next largest in location

12, and the next largest in location 2. The first group consists of P.U.s 2, 5, 8, 12, and 13.

Continue, to get the remaining two groups.

2. Suppose P.U. #7 is put into group 3.

Size of group 3 is 1,000 beds and size of P.U. #7 is 100 beds.

P.U. #7 will be selected from group 3 with probability 100/1000 = .1.

Example 8. In a particular region of the U.S. there are N = 90 universities with government

research grants. Because these universities are so widespread, it was decided to use a sample of

n = 10 universities. Rather than audit all grants at a selected university, it was decided (based on

available resources) to audit roughly 20% of the grants at each selected university. We know

that there are a total of M = 4,500 grants in all 90 universities.

Size: As a measure of the size for each university, use the total grant dollars.

Each row of the data file will contain:

University ID, number of grants, total grant dollars

i.e.,

ID of P.U., number of S.U.s (universe) in this P.U., size of P.U.

There are 90 rows of data (one for each P.U.) contained in this data set, named UNIVRHC.TXT.

Output: The 10 universities to use in the sample (see last page of computer output) are:

UNIV78, UNIV42, UNIV49, UNIV5, UNIV19, UNIV38, UNIV62, UNIV28,UNIV60, and UNIV75

This program will create an output file specified by the user (OutRHCsummary.txt for this

illustration) that is used as one of the input files by the RHC appraisal program. Dataset


(Rev. 10/2004) Page 1-13

UNIVRHC.TXT, the program output, and output file OutRHCsummary.txt are contained in the

following pages.

Dataset UNIVRHC.TXT < - - - continued - - - > (1) (2) (3)

< - - - continued - - - >

UNIV1 42 8UNIV2 21 4UNIV3 63 13UNIV4 74 16UNIV5 51 11UNIV6 43 9UNIV7 57 11UNIV8 49 10UNIV9 63 13UNIV10 18 4UNIV11 64 13UNIV12 56 11UNIV13 19 4UNIV14 44 9UNIV15 20 4UNIV16 34 7UNIV17 25 6UNIV18 38 9UNIV19 72 16UNIV20 46 10UNIV21 44 9UNIV22 64 13UNIV23 45 9UNIV24 55 11UNIV25 29 7UNIV26 36 7UNIV27 40 9UNIV28 78 18UNIV29 49 10UNIV30 60 12



Columns: (1) primary unit ID(2) number of grants(3) grant dollar amount (x $100,000) µ This is the size of the university.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/15/2004 GENERATION OF PRIMARY UNIT SAMPLE Time: 12:52 NAME OF INPUT FILE: C:\TEMP\UNIVRHC.TXT

GROUPS OF PRIMARY UNITS


Page 1-14 (Rev. 10/2004)

********* GROUP 1 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV51 11 55 UNIV44 17 76 UNIV32 14 66 UNIV78 <-- selected 7 37 UNIV79 18 77 UNIV2 4 21 UNIV52 9 38 UNIV33 5 25 UNIV47 5 24

GROUP TOTALS: 9 90 419

********* GROUP 2 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV6 9 43 UNIV42 <-- selected 12 63 UNIV65 7 32 UNIV40 18 77 UNIV45 10 51 UNIV1 8 42 UNIV80 10 49 UNIV36 5 24 UNIV70 7 31




********* GROUP 4 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV73 4 23 UNIV50 10 49 UNIV58 11 57 UNIV57 4 23 UNIV82 14 66 UNIV23 9 45 UNIV5 <-- selected 11 51 UNIV64 9 46


(Rev. 10/2004) Page 1-15

UNIV34 12 60


********* GROUP 5 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV84 17 77 UNIV35 4 19 UNIV16 7 34 UNIV81 17 76 UNIV27 9 40 UNIV85 6 27 UNIV19 <-- selected 16 72 UNIV68 6 28 UNIV26 7 36


********* GROUP 6 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV37 9 44 UNIV83 6 28 UNIV63 7 31 UNIV14 9 44 UNIV43 11 52 UNIV31 11 52 UNIV15 4 20 UNIV48 15 68 UNIV38 <-- selected 17 76


********* GROUP 7 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV69 14 66 UNIV18 9 38 UNIV25 7 29 UNIV59 11 53 UNIV30 12 60 UNIV10 4 18 UNIV24 11 55 UNIV62 <-- selected 18 77 UNIV17 6 25



Page 1-16 (Rev. 10/2004)

********* GROUP 8 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV28 <-- selected 18 78 UNIV41 8 37 UNIV89 15 71 UNIV66 14 68 UNIV11 13 64 UNIV86 17 75 UNIV56 12 59 UNIV12 11 56 UNIV72 7 33


********* GROUP 9 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV71 6 27 UNIV8 10 49 UNIV67 9 41 UNIV3 13 63 UNIV60 <-- selected 13 64 UNIV76 10 47 UNIV74 15 71 UNIV9 13 63 UNIV20 10 46


********* GROUP 10 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV22 13 64 UNIV39 9 41 UNIV88 12 59 UNIV55 15 71 UNIV29 10 49 UNIV75 <-- selected 16 75 UNIV87 15 71 UNIV13 4 19 UNIV53 16 72



(Rev. 10/2004) Page 1-17

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/15/2004 GENERATION OF PRIMARY UNIT SAMPLE Time: 12:52 NAME OF OUTPUT FILE: C:\TEMP\OutRHCsummary.txt

FIRST SEED NUMBER: 100.00 SECOND SEED NUMBER: 200.00

NUMBER OF PRIMARY UNITS IN THE POPULATION: 90 NUMBER OF PRIMARY UNITS SAMPLED: 10

UNITS SECONDARY PRIMARY INPRIMARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP========================= ============= ============= ============= =====UNIV78 37 7 90 9UNIV42 63 12 86 9UNIV49 34 7 96 9UNIV5 51 11 84 9UNIV19 72 16 89 9UNIV38 76 17 89 9UNIV62 77 18 92 9UNIV28 78 18 115 9UNIV60 64 13 99 9UNIV75 75 16 110 9

NOTE: In practice, it is recommended that you not set the two seed values unless you aretrying to duplicate prior results.

Output file OutRHCsummary.txt

(1) (2) (3) (4) (5)UNIV78 37 7 90 9UNIV42 63 12 86 9UNIV49 34 7 96 9UNIV5 51 11 84 9UNIV19 72 16 89 9UNIV38 76 17 89 9UNIV62 77 18 92 9UNIV28 78 18 115 9UNIV60 64 13 99 9UNIV75 75 16 110 9

Columns: (1) selected primary unit(2) number of grants (secondary units)(3) grant dollar amount (x $100,000) µ This is the size of the primary unit.(4) size of the group containing this primary unit(5) number of universities (primary units) in this group

RHC Three-Stage Sample Selection RAT-STATS Companion Manual

Page 1-18 (Rev. 10/2004)

Procedure for Three-Stage RHC Sampling

1. A sample of primary units (clusters) is obtained as in the two-stage procedure. The size

of the primary units is considered for this sample, where pps sampling is used for each

group of primary units.

2. A sample of secondary units is obtained within each chosen primary unit by partitioning

the primary unit into random groups of secondary units. The numbers of S.U.s in each

group are chosen to be as nearly equal as possible. Using pps sampling, and the size of

each secondary unit, one secondary unit is chosen from each of the secondary groups.

3. A random sample of third-stage units is obtained for each of the chosen secondary units.

No attention is paid to “size” here. This is a random sample.

Example 9. The previous example was expanded to include geographical regions.

Primary units: 12 regions (select four)

Secondary units: Universities (select 10 from each region)

Third stage units: Grants (audit 20% from each university)

Selection of Primary Units

A file must be constructed containing (for each region) (1) the number of secondary units

(universities) in this region and (2) the size of this region (total grant dollars). This file is

GRANTSPU.TXT. The selected regions are 4, 6, 8, and 10 using seed values of 100 and 200.

RAT-STATS Companion Manual RHC Three-Stage Sample Selection

(Rev. 10/2004) Page 1-19

File GRANTSPU.TXT

(1) (2) (3) REGION1 117 1250 REGION2 63 610 REGION3 91 720 REGION4 123 1320 REGION5 107 1160 REGION6 116 1240 REGION7 102 960 REGION8 118 1300 REGION9 122 1320 REGION10 85 640 REGION11 94 930 REGION12 62 550

Columns: (1) region ID(2) number of universities (secondary units)(3) size (total grant amount x $100,000)

Selection of Secondary Units

The three-stage RHC sample selection procedure requires the user to only obtain information for

each selected primary unit (i.e., regions 4, 6, 8, and 10 here). The information in each of these

four files consists of the size of each secondary unit (university, here) and the number of third-

stage units in the universe for each secondary unit. Each of these files should resemble file

UNIVRHC.TXT contained in the previous two-stage RHC discussion. Consequently, for each

sampled P.U., each line of the corresponding file should contain:

university ID, number of grants at this university, total grant dollars

i.e.,

secondary unit ID, no. of third-stage units, size of S.U.

After running the RHC Sample Selection program on each of these four regions, the following

universities were selected:


Page 1-20 (Rev. 10/2004)

REGION UNIVERSITIES 4 85, 46, 7, 82, 30, 34, 27, 66, 65, 80

6 113, 43, 78, 104, 89, 112, 30, 65, 3, 99

8 112, 6, 77, 93, 75, 111, 62, 115, 70, 99

10 78, 43, 7, 73, 55, 33, 10, 59, 64, 39

Selection of Third-Stage Units

Suppose that approximately 20% of the grants at each selected university are to be audited. Each

of these 40 samples (4 regions x 10 universities) is obtained randomly using the Single-Stage

Random Numbers program.

NOTES: (1) The previous five program runs (one at the primary level and four at thesecondary level) created five output files. Using a word processor, these files canbe joined to form one of the input files (the one containing primary/secondary unitinformation) for the three-stage RHC appraisal program which calculates theconfidence interval.

(2) This example is examined in more detail in the three-stage RHC appraisal section.

Summary of Input for RHC Sample Selection

RHC Two-Stage

The user must know:

1. The number of P.U.s in the universe and the sample.

2. The size of all P.U.s in the universe.

Procedure: 1. The user can set the number of S.U.s in each universe P.U. equal to one ifthese are difficult to determine. This is the middle column in fileUNIVRHC.TXT used in the previous illustration.

2. Next, run the RHC Two-Stage Sample Selection program. Store the outputin a text file.


(Rev. 10/2004) Page 1-21

3. Using a word processor or spreadsheet, change the number of S.U.s for eachsampled P.U. from one to the correct value.

RHC Three-Stage

The user must know:

1. The number of P.U.s in the universe and the sample.

2. The size of all P.U.s in the universe.

3. The number of S.U.s to be sampled within each P.U.

4. For each sampled P.U.,

a. The number of S.U.s in the universe within this P.U.

b. The size of all S.U.s within this P.U.

Procedure: 1. The user can set the number of S.U.s in each universe P.U. equal to 1 if theseare difficult to determine.

2. Run the RHC Two-Stage Sample Selection program. Store the output in atext file.

3. Using a word processor or spreadsheet, change the number of S.U.s for eachsampled P.U. from 1 to the correct value.

4. For each sampled P.U., build a data file where each row consists ofa. S.U. IDb. Number of third-stage units for this S.U. (OK to use a value of 1 here and

correct later).c. Size of this S.U.

5. For each sampled P.U., use the data set in step 4 as input to the RHC Two-Stage Sample Selection program. Store the output in a text file.

6. Using a word processor or spreadsheet, change the number of third-stage unitsfor each sampled S.U. from 1 to the correct value.

7. Merge the results from step 2 and each sampled P.U. into one file. SeePUSURHC3.TXT (below) for an example. The values in the second column(123, 54, 44, . . .) can be set to one and later changed to the correct values. This is one of the input files to the RHC Three-Stage Appraisal program.


Page 1-22 (Rev. 10/2004)

NOTE: Although this procedure allows for substituting 1s for the number ofsecond- and third-stage units in the original pass, the required sizeinformation must be known.

8. Build the data file (file PUSURHC3.TXT for this illustration) containing thesampled third-stage units. This is the other input file required by the RHCthree-stage appraisal program. Using a word processor or spreadsheet, thecolumn of sample sizes (highlighted) was added to the files created by the fiveRHC Sample Selection programs.

File PUSURHC3.TXT

REGION4 123 10 1320 3410 3UNIV85 54 11 11 125 12UNIV46 44 9 9 131 12UNIV7 77 15 17 119 12UNIV82 52 10 11 129 12UNIV30 54 11 11 141 12UNIV34 50 10 10 140 12UNIV27 76 15 16 138 12UNIV66 76 15 16 128 13UNIV65 62 12 14 125 13UNIV80 70 14 15 155 13REGION6 116 10 1240 3100 3UNIV113 33 7 8 108 11UNIV43 39 8 7 105 11UNIV78 63 13 12 104 11UNIV104 25 5 9 96 11UNIV89 35 7 7 124 12UNIV112 27 5 10 108 12UNIV30 58 12 11 95 12UNIV65 57 11 10 109 12UNIV3 56 11 11 115 12UNIV99 80 16 14 113 12REGION8 118 10 1300 3170 3UNIV112 75 15 16 125 11UNIV6 34 7 8 127 11UNIV7 51 10 11 120 12UNIV93 54 11 11 136 12UNIV75 52 10 11 126 12UNIV111 84 17 17 134 12UNIV62 64 13 14 123 12UNIV115 59 12 13 137 12UNIV70 65 13 14 143 12UNIV99 60 12 13 129 12REGION10 85 10 640 2320 3UNIV78 39 8 7 62 8UNIV43 42 8 7 68 8


(Rev. 10/2004) Page 1-23

File PUSURHC3.TXT - continued

UNIV7 56 11 9 54 8UNIV73 27 5 5 63 8UNIV55 78 16 12 70 8UNIV33 65 13 10 77 9UNIV10 60 12 9 76 9UNIV59 52 10 8 71 9UNIV64 50 10 8 73 9UNIV39 38 8 6 68 9

Generating Spares for RHC RAT-STATS Companion Manual

Page 1-24 (Rev. 10/2004)

Generating Spares for RHC Sampling

One question that arises here is what to do if one or more of the selected primary units is

unattainable or unusable for some reason. There is a method of generating spares without

having to start the sample selection process all over again, once the nonusable primary units

have been identified.

A similar approach can be used in a three-stage plan if one or more secondary units are

nonusable within a selected primary unit. The following example will illustrate how to recover

when one or more primary units are nonusable with a two-stage RHC sampling plan.

Example 10. Population: N = 90 P.U.s (universities) Sample: n = 5 P.U.s

The final section of the output using the RHC sample selection program is shown below;

it turns out that universities 51 (in group 5) and 69 (in group 4) could not be used.

UNITS SECONDARY PRIMARY INPRIMARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP========================= ============= ============= ============= =====UNIV47 24 5 191 18UNIV64 46 9 175 18UNIV52 38 9 185 18UNIV69), 66 14 203 18UNIV51)2)) can't use 55 11 196 18

The corresponding output file created from the first pass is shown below.

UNIV47 24 5 191 18 UNIV64 46 9 175 18 UNIV52 38 9 185 18 UNIV69 66 14 203 18 UNIV51 55 11 196 18

The section of the output containing the contents of group 4 follows.

RAT-STATS Companion Manual Generating Spares for RHC

(Rev. 10/2004) Page 1-25

********* GROUP 4 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV15 4 20 UNIV81 17 76 UNIV38 17 76 UNIV59 11 53 UNIV13 4 19 UNIV76 10 47 UNIV20 10 46 UNIV66 14 68 UNIV14 9 44 UNIV29 10 49 UNIV55 15 71 UNIV26 7 36 UNIV17 6 25 UNIV87 15 71 UNIV22 13 64 UNIV50 10 49 UNIV84 17 77 UNIV69 <-- Selected, can't use 14 66


Remove UNIV69 from the population and this group. Construct a data file (same format as

UNIVRHC.TXT) using this group only. This file (TEMP1.TXT) is shown below.

NOTE: When constructing this file, notice that columns 2 and 3 above (i.e., PRIMARYUNIT SIZE and SECONDARY UNIVERSE) need to be switched. This was donecorrectly in TEMP1.TXT.

UNIV15 20 4UNIV81 76 17UNIV38 76 17UNIV59 53 11UNIV13 19 4UNIV76 47 10UNIV20 46 10UNIV66 68 14 File TEMP1.TXTUNIV14 44 9UNIV29 49 10UNIV55 71 15UNIV26 36 7UNIV17 25 6UNIV87 71 15UNIV22 64 13UNIV50 49 10UNIV84 77 17

Generating Spares for RHC RAT-STATS Companion Manual

Page 1-26 (Rev. 10/2004)

Next, run the RHC Sample Selection program. Your input file is TEMP1.TXT and your sample

size is 1. This generates another P.U. (university) from this group. The output from this

program is shown below. UNIV22 was selected.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/15/2004 GENERATION OF PRIMARY UNIT SAMPLE Time: 13:17 NAME OF INPUT FILE: C:\TEMP\TEMP1.txt


********* GROUP 1 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV81 17 76 UNIV76 10 47 UNIV15 4 20 UNIV20 10 46 UNIV59 11 53 UNIV13 4 19 UNIV50 10 49 UNIV87 15 71 UNIV22 <--- Selected 13 64 UNIV29 10 49 UNIV84 17 77 UNIV55 15 71 UNIV26 7 36 UNIV66 14 68 UNIV38 17 76 UNIV14 9 44 UNIV17 6 25


NAME OF OUTPUT FILE: C:\TEMP\OUTTEMP1.TXT



UNITS SECONDARY PRIMARY INPRIMARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP========================= ============= ============= ============= =====UNIV22 64 13 189 17

RAT-STATS Companion Manual Generating Spares for RHC

(Rev. 10/2004) Page 1-27

Next, repeat this for group 5. Remove UNIV51 from the population and this group. Construct

data file TEMP2.TXT.

NOTE: As before, be sure to switch columns 2 and 3 when building this file.

UNIV63 31 7 UNIV18 38 9 UNIV58 57 11 UNIV31 52 11 UNIV56 59 12 UNIV90 72 16 UNIV65 32 7 File TEMP2.TXT UNIV12 56 11 UNIV16 34 7 UNIV2 21 4 UNIV79 77 18 UNIV74 71 15 UNIV8 49 10 UNIV86 75 17 UNIV53 72 16 UNIV23 45 9 UNIV33 25 5

Again, run the RHC Sample Selection program. The input file is TEMP2.TXT and the sample

size is 1. This generates another P.U. from this group. The output from this program is shown

below. UNIV86 was selected.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/15/2004 GENERATION OF PRIMARY UNIT SAMPLE Time: 13:24 NAME OF INPUT FILE: C:\TEMP\Temp2.txt


********* GROUP 1 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV18 9 38 UNIV90 16 72 UNIV63 7 31 UNIV65 7 32 UNIV31 11 52 UNIV56 12 59 UNIV23 9 45 UNIV86 <--- Selected 17 75 UNIV53 16 72 UNIV2 4 21 UNIV33 5 25

RHC Versus SRS RAT-STATS Companion Manual

Page 1-28 (Rev. 10/2004)

< OUTPUT -- continued >

UNIV79 18 77 UNIV74 15 71 UNIV12 11 56 UNIV58 11 57 UNIV16 7 34 UNIV8 10 49


NAME OF OUTPUT FILE: C:\TEMP\OUTTEMP2.TXT



UNITS SECONDARY PRIMARY INPRIMARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP========================= ============= ============= ============= ===== UNIV86 75 17 185 17

Finally, be sure to update the original output file shown earlier to reflect the two new selected

universities. This is shown below. This file is one of the input files to the RHC Two-Stage

Appraisal program.

Final output file (input file to RHC appraisal program)

UNIV47 24 5 191 18 UNIV64 46 9 175 18 UNIV52 38 9 185 18 UNIV22 64 13 189 17 UNIV86 75 17 185 17

Discussion: RHC Three-Stage sampling

A similar procedure can be used to generate “spare” secondary units. For example, if one of the

secondary units within a selected primary unit is nonusable, another secondary unit can be

selected from this group using the procedure outlined above.

RAT-STATS Companion Manual RHC Versus SRS

(Rev. 10/2004) Page 1-29

Discussion: Final-stage units

At the second stage for RHC Two-Stage sampling and the third stage for RHC Three-Stage

sampling, a random sample of units is obtained. Spares for this stage can be obtained in the

usual manner using the single-stage random number generator software (Single Stage Random

Numbers).

Comparison of RHC and Multistage SRS

In general, you can expect greater precision with the RHC procedure, provided there is a

significant correlation between the second and third columns (Number of Units and Size of Unit)

of each file using the RHC sample selection procedure. To illustrate, consider the file containing

the primary unit information used in the three-stage RHC illustration.

(1) (2) (3) REGION1 117 1250 REGION2 63 610 Columns: (1) unit ID REGION3 91 720 (2) number of units . (3) size of unit . . REGION10 85 640 REGION11 94 930 REGION12 62 550

For this example, the correlation between Size of Unit and Number of Units is .958, and we

would expect a two-stage RHC procedure to work quite well. For a three-stage procedure, this

correlation rule must also apply within each of the sampled primary units, at the secondary unit

level.

RHC Versus SRS RAT-STATS Companion Manual

Page 1-30 (Rev. 10/2004)

The benefits of RHC sampling include:

• increased precision if the above correlation rule is satisfied;

• maintaining the flavor of pps sampling, since pps sampling is used to select a unit from

each random group;

• relatively simple and straightforward computations;

• unbiased and stable point estimate of the universe total . This implies that when( $)T

sampling indefinitely, , on the average, is equal to the actual universe total, T, and$T

will exhibit relatively small variation.$T

• a staple point estimate of the variance of , producing more reliable confidence$T

intervals. This implies that when sampling indefinitely, the lower confidence limits will

exhibit relatively small variation.

RAT-STATS Companion Manual Attribute Appraisals

(Rev. 10/2004) Page 2-1

# Unrestricted

# Stratified

# Two-Stage Unrestricted

# Three-Stage Unrestricted

# RHC Two Stage

# RHC Three Stage

# Stratified Cluster

# Stratified Multistage

ATTRIBUTE APPRAISALS

An attribute appraisal is carried out to estimate a particular universe proportion (p) and its

corresponding sampling error. This proportion is typically an error rate (proportion of the

universe in error) but, more generally, it is the proportion of the universe items that meet (or do

not meet) a specified set of criteria. Also of interest may be the total number of items in the

universe (Np) that meet the criteria.

In an attribute sample, each sample item is either a yes response (met the criteria) or no response

(did not meet the criteria). This version of RAT-STATS contains eight modules that can be used

to appraise an attribute sample. These sampling strategies are listed below and described in the

sections to follow.

Unrestricted Attribute Appraisal RAT-STATS Companion Manual

Page 2-2 (Rev. 10/2004)

Unrestricted Attribute Appraisal

An unrestricted sample is the same as a simple random sample. Consequently, every sample

of size n has the same chance of being selected. For an unrestricted sample, a sample of size n is

randomly obtained and the number of sample elements meeting the criteria (say, x) is recorded.

Example 1. An unrestricted sample of 400 documents was obtained and examined to determine

if they had the proper approval signature. In the sample, 82 of the items did not contain the

proper signature (were in error). The sample error rate is then 82/400 = .205 (i.e., 20.5%). This

is the estimate of p, the error rate for the entire universe. If the universe size is N = 10,000, then

the estimated number of universe items in error is (10,000)(.205) = 2,050 items.

Using the RAT-STATS software, the corresponding 90% confidence interval for the total

number of universe items in error is from 1,729 to 2,403. The 90% confidence interval for the

universe error rate (p) is from 17.29% to 24.03%. Notice that the (point) estimate of 20.5% is

between 17.29% and 24.03% but it is not in the center of this interval. The center of the 90%

confidence interval is (17.29 + 24.03)/2 = 20.66%. The reason for this result is that this

estimation procedure is based on the exact hypergeometric distribution, rather than the normal

approximation. The resulting 95% confidence interval for p is 16.73% to 24.70% and for Np

(the total number of errors in the universe) is from 1,673 to 2,470.

RAT-STATS Companion Manual Unrestricted Attribute Appraisal

(Rev. 10/2004) Page 2-3

Discussion. Consider the 90% confidence interval. Define

TAIL = (1 - .90) / 2 = .05

The 90% confidence interval for Np is, say, k1 to k2. There were x = 82 sample items in error, so

(referring to the Formulas section below) k1 is the smallest value of k for which the probability

of observing 82 or fewer errors is > .05, where .05 is the value of TAIL. This value of k is

k1 = 1,729. The corresponding error rate is 1,729/10,000 = .1729 (i.e., 17.29%). To find the

upper limit of the 90% confidence interval, the program determines the largest value of k (say,

k2) for which the probability of observing x = 82 or more errors is > TAIL = .05. This is k2 =

2,403 with a corresponding error rate of 2,403/10,000 = .2403 (i.e., 24.03%). A similar

argument applies to the 95% confidence interval, where now the value of TAIL is .025.

NOTES: 1. Using these definitions of k1 and k2 for a 90% confidence interval, the user can beassured that the actual confidence level is at least 90%. This also applies to 80%and 95% confidence intervals.

2. In the event that no items having the characteristic(s) of interest are found in thesample, the user has the option of having the program determine both confidencelimits or only the upper confidence limits.

3. In the event that the number of items having the characteristic(s) of interest in thesample is the same as the sample size, the user has the option of having theprogram determine both confidence limits or only the lower confidence limits.

4. The universe size (N) is declared to be a long integer in the RAT-STATS

program. Consequently, the largest allowable universe size is N = =2 131 −2,147,483,647.

Unrestricted Attribute Appraisal RAT-STATS Companion Manual

Page 2-4 (Rev. 10/2004)

FORMULAS

To determine a 90% confidence interval for the total number of universe items in error, define

TAIL = (1 - .90)/2 = .05.

Upper Limit: Let k2 = largest value of k for which

ki

N kn iNn

i

x⎛⎝⎜

⎞⎠⎟

−−

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

>=∑

005.

where N = universe size

n = sample size

k = total number of universe items in error

x = number of sample items in error

Lower Limit: Let k1 = smallest value of k for whichki

N kn iNn

i x

n⎛⎝⎜

⎞⎠⎟

−−

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

>=∑ .05

The resulting 90% confidence interval for the total number of universe items in error is from k1

to k2 and the corresponding 90% confidence interval for the error rate (p) is k1/N to k2/N.

For a 95% confidence interval, use the same two equations, where .05 is replaced with

TAIL = .025. For an 80% confidence interval, the value of TAIL is .10.

The procedure used to derive this confidence interval can be found in the following article.

John P. Buonaccorsi (1987), “A Note on Confidence Intervals for Proportions in Finite

Populations,” The American Statistician, Vol. 41, No. 3, 215-218.

RAT-STATS Companion Manual Unrestricted Attribute Appraisal

(Rev. 10/2004) Page 2-5

Standard Errors

For universe proportion: Standard Error = where = x/n.$( $)p p

nnN

11

1−−

−⎛⎝⎜

⎞⎠⎟

$p

For universe total: Standard Error = .N p pn

nN

•$( $)1

11−

−−⎛

⎝⎜⎞⎠⎟

NOTE: RAT-STATS does not use the preceding standard errors when deriving a confidenceinterval for the universe proportion and universe total. Other software packages usethis standard error to derive an approximate confidence interval based on the normaldistribution. RAT-STATS derives an exact confidence interval based on thehypergeometric distribution.

Stratified Attribute Appraisal RAT-STATS Companion Manual

Page 2-6 (Rev. 10/2004)

Stratified Attribute Appraisal

In a stratified attribute sampling plan, the universe is divided into two or more nonoverlapping

categories (strata). As with an unrestricted sample, the intent is to make a statistical estimate for

a universe proportion (p) or a universe total (Np) that meets a specified set of criteria. This plan

involves obtaining a random sample from each of the strata. The program will request the

number of universe items in each stratum and these values must be known. The program will

develop estimates for each stratum as well as for the entire universe.

NOTE: In the discussion to follow, we will refer to the proportion, p, as the “error rate.”

Example 2. A universe of 2,500 Medicare claims is stratified into inpatient (Stratum 1) and

outpatient (Stratum 2) claims. The universe sizes are N1 = 1,000 inpatient claims and N2 = 1,500

outpatient claims. Of interest is the proportion, p, of claims in error (containing improper

charges).

A random sample of n1 = 100 inpatient claims revealed x1 = 2 errors and a random sample of

n2 = 100 outpatient claims uncovered x2 = 6 errors.

NOTE: Both random samples were obtained using the Single-Stage Random Numbersprogram whereby 100 random numbers between 1 and 1,000 were obtained forstratum 1 and 100 random numbers between 1 and 1,500 were obtained for stratum 2.

RAT-STATS Companion Manual Stratified Attribute Appraisal

(Rev. 10/2004) Page 2-7

The following output was obtained from the stratified attribute appraisal program.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 2/7/2004 STRATIFIED ATTRIBUTE APPRAISAL Time: 10:55 AUDIT/REVIEW: Attribute - Stratified

PROJ. ITEMSSTRATUM SAMPLE *ITEMS** **RATIO* *UNIVERSE* IN UNIVERSE======= ====== ======== ======== ========== ===========

1 100 2 2.000% 1,000 20 2 100 6 6.000% 1,500 90

COMBINED 200 8 4.400% 2,500 110

STANDARD ERROR: 1.483% 37

PRECISION PRECISION PRECISIONSTRATUM AT 80% CL AT 90% CL AT 95% CL======= ========= ========= ========= 1 1.711% 2.196% 2.616% 2 2.955% 3.793% 4.519%

COMBINED 1.901% 2.439% 2.907%

LOWER LIMIT - QUANTITY 62 49 37 PERCENT 2.499% 1.961% 1.493%UPPER LIMIT - QUANTITY 158 171 183 PERCENT 6.301% 6.839% 7.307%

Discussion. The strata sample error rates are 2% and 6%. The projected number of inpatient

claims in error is (.02)(1,000) = 20 and the projected number for the outpatient stratum is

(.06)(1,500) = 90. Consequently, the projected value for the universe is 20 + 90 =110

(highlighted) with a corresponding error rate of (110/2,500) x 100% = 4.4% (highlighted).

A look at the inpatient stratum: The estimated error rate is 2%. The corresponding precision

at the 90% confidence level is 2.196% (highlighted). The term “precision” refers to the amount

that is added and subtracted to the point estimate (2%, here) in deriving a confidence interval.


Page 2-8 (Rev. 10/2004)

Consequently, the 90% confidence interval for the proportion of inpatient claims in error is 2% ±

2.196%; that is, -0.196% to 4.196%. Since the lower limit is negative, it may be set equal to

zero. Similarly, the 95% confidence interval for the proportion of inpatient claims in error is

2% ± 2.616% (highlighted); that is, 0% to 4.616%, once again setting the lower limit equal to

zero.

NOTE: These confidence intervals are not actually contained in the program output.

A look at the outpatient stratum: The estimated error rate is 6%. Continuing the discussion

from the inpatient stratum, the 90% confidence interval for the proportion of outpatient claims in

error is 6% ± 3.793% (highlighted); that is, 2.207% to 9.793%. The corresponding 95%

confidence interval is 6% ± 4.519% (highlighted); that is, 1.481% to 10.519%. As before, these

confidence intervals are not actually contained in the program output.

A look at the overall precision: The precision at the 90% level is 2.439% (highlighted) and so

the resulting 90% confidence interval for the universe proportion of claims in error is

4.4% ± 2.439%; that is, 1.961% to 6.839%. Multiplying these two values by 2,500 (and dividing

by 100), the corresponding 90% confidence interval for the total number of universe claims in

error is 49 to 171. Notice that these values are rounded to the nearest integer. Using the

precision at the 95% confidence level (i.e., 2.907%), the 95% confidence interval in the previous

output can be obtained.

RAT-STATS Companion Manual Stratified Attribute Appraisal

(Rev. 10/2004) Page 2-9

FORMULAS

The estimated proportion for stratum i is where and where xi is the number of$pi $ /p x ni i i=

sample elements in stratum i in error and ni is the number of sample items from stratum i. The

value of Ratio is x 100%. The Projected Items in Universe for stratum i is where$pi ( $ )( )p Ni i

Ni is the number of universe items in stratum i. The PRECISION AT 90% CL (CL stands for

confidence level) is 1.644853626951 times the standard error of ; that is$pi

16448536269511

1.

$ ( $ )N nN

p pn

i i

i

i i

i

−⋅

−−

To obtain the Precision at 95% (80%) CL value for the i-th stratum, replace 1.644853626951

with 1.959963984540 (1.281551565545). The estimated standard error of is$pi

SE pN n

Np p

nii i

i

i i

i( $ )

$ ( $ )=

−⋅

−−

11

Overall estimates: The estimate of the universe proportion (error rate) is (under the Ratio$p

heading), where

$ $pNN

pi

i

L

i=⎛⎝⎜

⎞⎠⎟

=∑

1

the summation is over all of the L strata and N = ENi is the total universe size. The estimated

standard error of is$p

SE pNN

SE pi

i

L

i( $) [ ( $ )]=⎛⎝⎜

⎞⎠⎟

=∑

1

22

The Precision at 90% CL value is 1.644853626951 A SE( ) .$p


Page 2-10 (Rev. 10/2004)

The Precision at 95% CL value is obtained by replacing 1.644853626951 by 1.959963984540 in

the above formula and for an 80% confidence interval, 1.644853626951 is replaced by

1.281551565545.

The resulting confidence intervals for the universe proportion (error rate) are

± (PRECISION)$p

To obtain the confidence intervals for the universe total, multiply both ends of the confidence

interval for the error rate by the universe size, N, and round to the nearest integer.

RAT-STATS Companion Manual Two-Stage Unrestricted Attribute Appraisal

(Rev. 10/2004) Page 2-11

Two-Stage Unrestricted

This is a special case of multistage sampling. Multistage sampling is a very cost-effective

sampling procedure when (1) obtaining a frame that lists all elements in the universe is

very costly or impossible, or (2) the cost of obtaining observations increases as the distance

separating the elements increases. Put another way, multistage sampling is cost effective

when it is more costly to get to the sampling unit than it is to audit the sampling unit. The goal

of multistage sampling is to get the most precise results per unit of examination cost.

General Comments

1. This is a very convenient sampling procedure for many situations because you don't have to

visit all the locations.

2. For a two-stage procedure, the universe can be broken down into “subgroups.”

Example: 1st Stage: Carriers (P.U.s)

2nd Stage: Hospitals (S.U.s)

So, the procedure is to first obtain a random sample of P.U.s. These are called clusters.

Then, obtain a random sample of S.U.s within each selected P.U.. Notice that at the first

stage, clusters are the sampling unit (sampling units are not always individual people,

records, etc.).

3. You can estimate cost overpayments for the entire universe with multistage sampling; it is

very useful for large, widespread universes.


research grants. Because these universities are so widespread, it was decided to use a two-stage

sample using 10 universities. Rather than audit all grants at a selected university, it was decided

Two-Stage Unrestricted Attribute Appraisal RAT-STATS Companion Manual

Page 2-12 (Rev. 10/2004)

(based on available resources) to audit roughly 20% of the grants at each selected university to

estimate the proportion of grants containing charges after the scheduled completion of the grant.

The following data were obtained, where ai (pi) is the number (proportion) of grants in the

sample from the i-th university containing such charges, mi is the number of audited (sampled)

grants at the i-th university, and Mi is the total number of grants in the audit universe at the i-th

university.

Univ. Mi mi ai pi Mipi 1 50 10 4 .400 20.00 2 65 13 5 .385 25.00 3 45 9 2 .222 10.00 4 48 10 3 .300 14.40 5 52 10 5 .500 26.00 6 58 12 3 .250 14.50 7 42 8 3 .375 15.75 8 66 13 4 .308 20.31 9 40 8 2 .250 10.00 10 56 11 4 .364 20.36 522 104 176.32

Define M to be the total number of secondary units (grants) in the universe. In practice, M may

be known or unknown. If M is unknown, the Two-Stage Unrestricted program estimates the

universe proportion using a ratio estimator. No estimate of the universe total (total number of

grants containing improper charges) is available. This is illustrated in the computer output to

follow where M is unknown.


(Rev. 10/2004) Page 2-13

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 1/31/2004 TWO STAGE UNRESTRICTED ATTRIBUTE APPRAISAL Time: 14:13 AUDIT/REVIEW: Example DATA FILE: C:\Temp\DATA2STG.TXT

PRIMARY SAMPLE ITEMS WITH UNIT UNIVERSE SAMPLE SIZE CHARACTERISTIC(S) RATIO======= ============ =========== ================= =====

1 50 10 4 40.00% 2 45 9 2 22.22% 3 52 10 5 50.00% 4 42 8 3 37.50% 5 40 8 2 25.00% 6 65 13 5 38.46% 7 48 10 3 30.00% 8 58 12 3 25.00% 9 66 13 4 30.77% 10 56 11 4 36.36%

TOTALS 522 104 35

TOTAL PRIMARY UNITS IN THE UNIVERSE 90

OVERALL RATIO 33.78%

STANDARD ERROR 2.85%

CONFIDENCE LEVEL 80 PERCENT 90 PERCENT 95 PERCENTLOWER LIMIT FOR PROPORTION 30.13% 29.09% 28.19%UPPER LIMIT FOR PROPORTION 37.43% 38.47% 39.36%

Discussion. The estimate of the universe proportion, p, is

= (estimated number of grants containing improper charges in the sampled universities) /$pr(number of grants in the sampled universities)

= [(50)(.4000) + (45)(.2222) + AAA + (56)(.3636)] / (50 + 45 + AAA + 56)

= 176.32 / 522 = .3378 (that is, 33.78%)

The estimated variance of is$pr

= (Standard Error)2 = (.0285)2 = .000812.v pr( $ )

NOTE: There is a formula for in the formula section.v pr( $ )


Page 2-14 (Rev. 10/2004)

The corresponding approximate 95% confidence interval for the universe proportion is

.3378 ± (Precision at 95% Confidence Level)

The Precision at 95% Confidence Level value is the same as 1.959963984540

times the (Standard Error). So, the resulting 95% confidence interval can also be written

.3378 ± (1.959963984540)(Standard Error)

.3378 ± (1.959963984540)(.0285)

.3378 ± .0559

.2819 to .3937 (28.19% to 39.36%).

NOTE 1: When the value of M is unknown in the formula for (the case here), it isv pr( $ )acceptable to replace this value with the average value of M for the sample (as wasdone in this illustration). This value is 522/10 = 52.2. This is an advantage of usingthis estimator, since it does not require knowledge of M. If M is known, the user hastwo choices: (1) use the above ratio estimator, where now M is known or (2) use anunbiased estimator of p, illustrated in Example 4.

NOTE 2: If the value of M is known, the RAT-STATS software uses the unbiased estimator,illustrated in Example 4.

Example 4. Suppose that in Example 3, it is known that there is a total number of M = 4,500

grants (secondary units) in all 90 universities. As a result, is known and is equal to M/N =M

4,500/90 = 50. The following output is obtained. Notice that estimated (projected) totals for

each sampled university (primary unit) and for the entire universe are provided.


(Rev. 10/2004) Page 2-15

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 1/31/2004 TWO STAGE UNRESTRICTED ATTRIBUTE APPRAISAL Time: 13:52 AUDIT/REVIEW: Example DATA FILE: C:\Temp\DATA2STG.TXT

PRIMARY SAMPLE ITEMS WITH UNIT UNIVERSE SAMPLE SIZE CHARACTERISTIC(S) RATIO PROJECTED======= ============ =========== ================= ===== =========

1 50 10 4 40.00% 20 2 45 9 2 22.22% 10 3 52 10 5 50.00% 26 4 42 8 3 37.50% 16 5 40 8 2 25.00% 10 6 65 13 5 38.46% 25 7 48 10 3 30.00% 14 8 58 12 3 25.00% 15 9 66 13 4 30.77% 20 10 56 11 4 36.36% 20

TOTALS 522 104 35

OVERALL TOTALS 90 4,500 35.26% 1,587

STANDARD ERROR 3.67% 165


LOWER LIMIT FOR TOTAL 1,375 1,315 1,263UPPER LIMIT FOR TOTAL 1,799 1,859 1,911

Discussion. An unbiased estimator of the universe proportion, p, is NM

AB

⎛⎝⎜

⎞⎠⎟⎛⎝⎜

⎞⎠⎟

where A = estimated number of grants containing improper charges in the sampleduniversities

B = number of sampled universities

and A/B = the projected average number of grants containing improper charges for thesampled universities

So, A/B = [(50)(.4000) + (65)(.3846) + AAA + (56)(.3636)] / 10

= 176.32 / 10 = 17.632 grants


Page 2-16 (Rev. 10/2004)

The projected number of grants containing improper charges for the universe is NA(A/B); that is,

(90)(17.632) = 1586.88 grants. Since there are 4500 grants in the universe, then the estimated

proportion of grants containing improper charges is 1586.88/4,500 = .3526 (35.26%).

The corresponding approximate 95% confidence interval for the universe proportion is

.3526 ± (Precision at 95% Confidence Level),

which is the same as .3526 ± 1.959963984540(Standard Error). So, the resulting 95%

confidence interval is

.3526 ± (1.959963984540)(Standard Error)

.3526 ± (1.959963984540)(.0367)

.3526 ± .0720

.2806 to .4247 (28.06% to 42.47%).

The corresponding 95% confidence interval for the total number of grants in the universe

containing improper charges is

1587 ± 324; that is, 1263 to 1911 grants.

NOTE: Formulas for and the corresponding confidence interval are contained in the$pu formula section.

FORMULAS

Case 1: When the total number of secondary units in the universe (M) is unkown, the ratio

estimator for the universe proportion is used. This estimator will be called . Define:$pr

Mi = number of secondary units in the universe for the i-th sampled primary unit, mi ofwhich are sampled


(Rev. 10/2004) Page 2-17

= proportion of secondary units having the attribute of interest in the i-th sampled$piprimary unit

n = number of sampled primary units

N = number of primary units in the universe (must be known)

M = number of secondary units in the universe (may be known or unknown)

= average number of secondary units per primary units in the universe. This is equal toMM/N if M is known. It can be estimated using if M is unknown, where is them maverage number of secondary units in the sampled primary units.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

The estimate of the universe proportion having the attribute of interest is

$

$

p

M p

Mr

i ii

n

ii

n= =

=

∑

∑1

1

The estimated variance of is $pr

v pN n

N nM

M p p

n nNMM

M mM

p pmr

i i ri

n

ii i

ii

ni i

i( $ )

( $ $ )$ ( $ )

=−⎛

⎝⎜⎞⎠⎟⎛⎝⎜

⎞⎠⎟

−

−

⎛

⎝

⎜⎜⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟⎟⎟

+−⎛

⎝⎜

⎞⎠⎟

−−

=

=

∑∑1

11 1

12

2 2

12

2

1

NOTE: The standard error of is the square root of .$pr v pr( $ )


Page 2-18 (Rev. 10/2004)

Case 2: When the total number of secondary units in the universe (M) is known, an unbiased

estimator for the universe proportion is used. This estimator will be called .$pu

$

$

pNM

M p

nu

i ii

n

= =∑

1

The estimated variance of is $pu

v pN n

N nM

M p Mp

n nNMM

M mM

p pmu

i i ui

n

ii i

ii

ni i

i( $ )

( $ $ )$ ( $ )

=−⎛

⎝⎜⎞⎠⎟⎛⎝⎜

⎞⎠⎟

−

−

⎛

⎝

⎜⎜⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟⎟⎟

+−⎛

⎝⎜

⎞⎠⎟

−−

=

=

∑∑1

11 1

12

2

12

2

1

NOTE 1: The standard error of is the square root of $pu v pu( $ ).

NOTE 2: When estimating the total number of secondary units in the universe having theattribute of interest, both and the standard error of are multiplied by M. $pu $puThe Precision at the 95% Confidence Level value for the universe total is(1.959963984540)(M)(standard error of ). For the Precision at the 90%$puConfidence Level value, replace 1.959963984540 with 1.644853626951 and for thePrecision at the 80% Confidence Level value, replace 1.959963984540 with1.281551565545.

RAT-STATS Companion Manual Three-Stage Unrestricted Attribute Appraisal

(Rev. 10/2004) Page 2-19

Three-Stage Unrestricted

Example 5. The situation discussed in Example 4 was extended the following year to a three

stage procedure by defining:

Stage 1: REGION (select 4 out of 12 regions)

Stage 2: UNIVERSITY (select 10 from each selected region)

Stage 3: GRANT (select approximately 20% of all grants at each university)

Using the random number module (Single-Stage Random Numbers), regions 5, 7, 8, and 10

were selected as the sampled primary units. Next, 10 universities (secondary units) were

randomly selected (again using program Single- Stage Random Numbers) from the available

universities in each of the four selected regions. The following data were obtained, where Mi is

the number of grants in the universe for each university, mi is the number of audited grants at

each university (chosen to be roughly 20% of Mi), and ai is the number of grants in the sample

from the i-th university containing charges after the scheduled completion of the grant (in error).

REGION 5 (contains 90 secondary units, 10 to be sampled)

Univ. Mi mi ai

1 47 9 3 2 51 10 2 3 45 9 4 4 46 9 1 5 46 9 3 6 50 10 1 7 50 10 4 8 57 11 3 9 54 11 4 10 64 13 2

Three-Stage Unrestricted Attribute Appraisal RAT-STATS Companion Manual

Page 2-20 (Rev. 10/2004)


Univ. Mi mi ai

1 53 11 2 2 59 12 5 3 52 10 1 4 67 13 3 5 59 12 1 6 73 15 6 7 51 10 3 8 75 15 2 9 66 13 1 10 58 12 4


Univ. Mi mi ai

1 45 9 3 2 39 8 2 3 43 9 4 4 34 7 1 5 54 11 2 6 54 11 3 7 34 7 1 8 59 12 1 9 49 10 4 10 43 9 2


Univ. Mi mi ai

1 59 12 2 2 68 14 6 3 57 11 3 4 72 14 6 5 70 14 1 6 73 15 2 7 83 17 5 8 89 18 4 9 73 15 3 10 77 15 2

The resulting data set in called DATA3ST.TXT and is shown on the next page. The

corresponding computer output using the Three-Stage Unrestricted program immediately

follows. For this illustration, the total number of third-stage units in the universe (S) is

unknown.


(Rev. 10/2004) Page 2-21

--- Data set DATA3ST.TXT ---

REGION 5 90 10 ))))))) There are 90 secondary units (universities) in thisUNIV1 47 9 3 primary unit (region); 10 were audited.UNIV2 51 10 2UNIV3 45 9 4UNIV4 46 9 1UNIV5 46 9 3UNIV6 50 10 1UNIV7 50 10 4UNIV8 57 11 3UNIV9 54 11 4UNIV10 64 13 2REGION 7 110 10UNIV1 53 11 2 ))))))) In UNIV1, there were 53 grants (third-stage units).UNIV2 59 12 5 Eleven of these grants were sampled and twoUNIV3 52 10 1 contained improper charges.UNIV4 67 13 3UNIV5 59 12 1UNIV6 73 15 6UNIV7 51 10 3UNIV8 75 15 2UNIV9 66 13 1UNIV10 58 12 4REGION 8 85 10UNIV1 45 9 3UNIV2 39 8 2UNIV3 43 9 4UNIV4 34 7 1UNIV5 54 11 2UNIV6 54 11 3UNIV7 34 7 1UNIV8 59 12 1UNIV9 49 10 4UNIV10 43 9 2REGION 10 120 10UNIV1 59 12 2UNIV2 68 14 6UNIV3 57 11 3UNIV4 72 14 6UNIV5 70 14 1UNIV6 73 15 2UNIV7 83 17 5UNIV8 89 18 4UNIV9 73 15 3UNIV10 77 15 2


Page 2-22 (Rev. 10/2004)

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 1/31/2004 THREE STAGE ATTRIBUTE APPRAISAL Time: 15:18 AUDIT/REVIEW: Example NAME OF INPUT FILE: C:\Temp\DATA3ST.TXT

FIRST STAGE NEXT STAGE MEETING SECOND STAGE UNIVERSE SAMPLE SIZE CRITERIA RATIO============================ ============ =========== ======== =======REGION 5 90 10 UNIV1 47 9 3 33.33% UNIV3 51 10 2 20.00% UNIV3 45 9 4 44.44% UNIV4 46 9 1 11.11% UNIV5 46 9 3 33.33% UNIV6 50 10 1 10.00% UNIV7 50 10 4 40.00% UNIV8 57 11 3 27.27% UNIV9 54 11 4 36.36% UNIV10 64 13 2 15.38%

TOTALS 510 101 27

REGION 7 110 10 UNIV1 53 11 2 18.18% UNIV2 59 12 5 41.67% UNIV3 52 10 1 10.00% UNIV4 67 13 3 23.08% UNIV5 59 12 1 8.33% UNIV6 73 15 6 40.00% UNIV7 51 10 3 30.00% UNIV8 75 15 2 13.33% UNIV9 66 13 1 7.69% UNIV10 58 12 4 33.33%

TOTALS 613 123 28

REGION 8 85 10 UNIV1 45 9 3 33.33% UNIV2 39 8 2 25.00% UNIV3 43 9 4 44.44% UNIV4 34 7 1 14.29% UNIV5 54 11 2 18.18% UNIV6 54 11 3 27.27% UNIV7 34 7 1 14.29% UNIV8 59 12 1 8.33% UNIV9 49 10 4 40.00% UNIV10 43 9 2 22.22%

TOTALS 454 93 23

REGION 10 120 10 UNIV1 59 12 2 16.67% UNIV2 68 14 6 42.86% UNIV3 57 11 3 27.27% UNIV4 72 14 6 42.86% UNIV5 70 14 1 7.14% UNIV6 73 15 2 13.33% UNIV7 83 17 5 29.41% UNIV8 89 18 4 22.22% UNIV9 73 15 3 20.00%


(Rev. 10/2004) Page 2-23

UNIV10 77 15 2 13.33%

TOTALS 721 145 34

OVERALL TOTALS UNIVERSE SAMPLED======= ====== ======== ======= FIRST STAGE 12 4 SECOND STAGE 405{} 40 THIRD STAGE 2,298{} 462 SAMPLED ITEMS MEETING CRITERIA 112 {} UNIVERSE SIZES FOR THE SECOND AND THIRD STAGES REPRESENT THE UNIVERSES FOR THE SAMPLED PRIOR STAGE.

OVERALL POINT ESTIMATE OF THE PROPORTION 24.06%OVERALL STANDARD ERROR (PROPORTION) 1.35%

OVERALL POINT ESTIMATE OF UNIVERSE TOTAL 17,210OVERALL STANDARD ERROR (TOTAL) 2,415



Highlighted values:

(1) 33.33% is 3/9(2) 405 = 90 (in Region 5) + 110 (in Region 7) + 85 (in Region 8) + 120 (in Region 10)(3) 2,298 = (47 + AAA + 64) in Region 5 + (53 + AAA + 58) in Region 7 + (45 + AAA + 43) in

Region 8 + (59 + AAA + 77) in Region 10.

Discussion. Based on the preceding output, the following results were obtained:

(1) Estimate of the proportion of grants in the universe containing improper charges is .2406

(24.06%). This uses Equation 1 in the Formulas and Definitions section.

(2) The 95% confidence interval for the universe proportion is from .2142 to .2670.

(3) Estimate of the total number of grants in the universe containing improper charges is

17,210. This uses Equation 5 in the Formulas and Definitions section.

(4) The 95% confidence interval for the total number of grants containing improper charges

is from 12,477 to 21,942.


Page 2-24 (Rev. 10/2004)

The standard error of the proportion estimate is .0135 (1.35%) and the 95% confidence interval

is .2406 ± (1.959963984540)(.0135); that is, .2142 to .2670. The value of .0135 is found by

taking the square root of the value obtained using Equation 2. For the universe total, the

standard error is 2,415 (using the square root of Equation 6) and the 95% confidence interval is

17,210 ± (1.959963984540)(2,415); that is, 12,477 to 21,942.

Is the total number of third-stage units in the universe known or unknown?

Let S = the total number of third-stage units in the universe. Two cases will be considered:

Case 1: S is unknown.

Case 2: S is known.

For Case 1:

To estimate the proportion, p, use the ratio (biased) estimator . To estimate the( $ )pr

number in the population (T) having the attribute of interest, use the unbiased estimator

.( $ )Tu

For Case 2:

To estimate the proportion, p, use the unbiased estimator ( ). To estimate the number$pu

in the population (T) having the attribute of interest, use the unbiased estimator ( ).$Tu

NOTE: In the preceding example, the unbiased estimator was used where = 17,210. $Tu $TuHere S was unknown and so the proportion estimator ( ) is from case 1 (Equation 1$prin the Formulas and Definitions section). The standard error of (.0135) is the$prsquare root of the value obtained from Equation 2.


(Rev. 10/2004) Page 2-25

FORMULAS

Definitions

S = total number of third-stage units in the universe

N = number of primary units in the universe

n = number of primary units in the sample

Mi = number of secondary units (universe) in i-th primary unit

mi = number of secondary units (sample) in i-th primary unit

Bij = number of third-stage units (universe) in j-th secondary unit within i-th primary unit

bij = number of third-stage units (sample) in j-th secondary unit within i-th primary unit

= proportion of bij sampled third-stage units in error$pij

Formulas for (Case 1) and (Case 2)$pr $pu

The ratio estimator (Equation 1)$pr

$

$

p

Mm

B p

Mm

B

r

i

iij ij

j

m

i

n

i

iij

j

m

i

n

i

i=

==

==

∑∑

∑∑11

11

Estimated variance of (Equation 2)$pr

v pN n

n n NST

TN

R BBNr i

ui

u

i

n

( $ )( )

$$

$ $$

=−

−−

⎛

⎝⎜

⎞

⎠⎟ − −

⎛

⎝⎜

⎞

⎠⎟

⎡

⎣⎢⎢

⎤

⎦⎥⎥=

∑1 21

2

+−−

−⎛

⎝⎜

⎞

⎠⎟ − −

⎛

⎝⎜

⎞

⎠⎟

⎡

⎣⎢⎢

⎤

⎦⎥⎥= =

∑ ∑112

1 1

2

nNSM M m

m mT

TM

R BBM

i i i

i ii

n

iji

iij

i

ij

mi( )( )

$$

$$

+−

−−

= =∑ ∑1

112

1 1nNSMm

B B bb

p pi

ii

nij ij ij

ijj

m

ij ij

i ( )$ ( $ )


Page 2-26 (Rev. 10/2004)

where $ $TNn

Tu ii

n

==∑

1

$ $TMm

B pii

iij ij

j

mi

==∑

1$ $T B pij ij ij=

$ $BNn

Bu ii

n

==∑

1

$BMm

Bii

iij

j

mi

==∑

1

SSN

=

$$

$RTB

u

u=

Notes: (1) In Equation 2, $ $R pr=

(2) To estimate use the sample estimate whereS , s

sBN n

Mm

Bu i

ii

n

ijj

mi

= == =∑ ∑

$ 1

1 1

The unbiased estimator (Equation 3)$pu

$ $pNnS

Mm

B pui

ii

n

ij ijj

mi

== =∑ ∑

1 1


(Rev. 10/2004) Page 2-27

Estimated variance of (Equation 4)$pu

This variance can most easily be determined by setting in equation 2. Consequently,$R = 0

v p N nn n NS

T TNu iu

i

n( $ )

( )$

$=

−

−−

⎛

⎝⎜

⎞

⎠⎟

=∑

1 21

2

+−−

−⎛

⎝⎜

⎞

⎠⎟

= =∑ ∑1

121 1

2

nNSM M m

m mT

TM

i i i

i ii

n

iji

ij

mi( )( )

$$

+−

−−

= =∑ ∑1

112

1 1nNSMm

B B bb

p pi

ii

nij ij ij

ijj

m

ij ij

i ( )$ ( $ )

The unbiased estimator (Equation 5)$Tu

Since then$ $T S pu u= ⋅

$ $TNn

Mm

B pui

ii

n

ij ijj

mi

== =∑ ∑

1 1

NOTE: The value of S is not needed here.

Estimated variance of (Equation 6)$Tu

Since , thenv T S v pu u( $ ) ( $ )= ⋅2

v TN N nn n

TTNu iu

i

n

( $ )( )( )

$$

=−−

−⎛

⎝⎜

⎞

⎠⎟

=∑1

1

2

+−−

−⎛

⎝⎜

⎞

⎠⎟

= =∑ ∑N

nM M m

m mT

TM

i i i

i ii

n

iji

ij

mi( )( )

$$

11 1

2

+−

−−

= =∑ ∑N

nMm

B B bb

p pi

ii

nij ij ij

ijj

m

ij ij

i

1 11

1( )

$ ( $ )

NOTE: The value of S is not needed here.

RHC Two Stage Attribute Appraisal RAT-STATS Companion Manual

Page 2-28 (Rev. 10/2004)

RHC Two Stage

For a discussion on the motivation behind the RHC sampling procedure, refer to the RHC

SAMPLE SELECTION section on page 1-11, contained in the RANDOM NUMBERS section

of this manual. It provides a method of sample selection that allows sampling without

replacement while “maintaining the flavor” of sampling using probability proportional to size.

When the P.U.s are selected, the size of each P.U. is considered rather than obtaining a simple

random sample of P.U.s.

The size of each P.U. is rather arbitrary and can be the number of people, dollars, beds (for

hospitals), and so forth. In general, you can expect improved precision using the RHC procedure

if there is a high correlation between the size of each P.U. and the number of S.U.s within each

P.U.. In other words, P.U.s having a larger size should contain a larger number of S.U.s.

The P.U.s are selected using the RHC SAMPLE SELECTION program. A random sample is

then obtained for each selected P.U. and the number of S.U.s having the attribute of interest

(e.g., in error) is recorded.

Example 6. An audit was carried out for state-supported university grants in a particular region.

The universe consisted of all charge vouchers recorded for these grants. Because these

universities are so widespread, it was decided to employ a two-stage sample using three of the 27

state-supported universities. Rather than audit all the vouchers at a selected university, it was

decided (based on available resources) to audit 250 vouchers at each selected university to

estimate the proportion of vouchers containing improper charges. The universities (P.U.s) were

RAT-STATS Companion Manual RHC Two Stage Attribute Appraisal

(Rev. 10/2004) Page 2-29

to be selected using the RHC procedure where the “size” of each university was the total grant

dollars awarded to that university.

The following file (RHC2STAGE.TXT) was constructed:

(1) (2) (3)UNIV1 14928 8 UNIV2 12454 4UNIV3 17404 13UNIV4 18700 16UNIV5 15989 11UNIV6 15046 9UNIV7 16696 11UNIV8 15754 10UNIV9 17404 13UNIV10 12100 4UNIV11 17522 13UNIV12 16578 11UNIV13 12218 4UNIV14 15164 9 Data file RHC2STAGE.TXTUNIV15 12336 4UNIV16 13986 7UNIV17 12925 6UNIV18 14457 9UNIV19 18464 16UNIV20 15400 10UNIV21 15164 9UNIV22 17522 13UNIV23 15282 9UNIV24 16461 11UNIV25 13396 7UNIV26 14222 7UNIV27 14693 9

Columns: (1) unit ID(2) number of vouchers (3) size of university (dollar amount of grants x $10,000)

Using the RHC SAMPLE SELECTION program, the following output is produced:


Page 2-30 (Rev. 10/2004)

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 GENERATION OF PRIMARY UNIT SAMPLE Time: 14:29 NAME OF INPUT FILE: C:\TEMP\RHC2STAGE.txt


********* GROUP 1 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV27 9 14,693 UNIV2 4 12,454 UNIV6 9 15,046 UNIV1 8 14,928 UNIV7 <-- Selected 11 16,696 UNIV21 9 15,164 UNIV4 16 18,700 UNIV5 11 15,989 UNIV16 7 13,986

GROUP TOTALS: 9 84 137,656

********* GROUP 2 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV19 16 18,464 UNIV20 <-- Selected 10 15,400 UNIV14 9 15,164 UNIV26 7 14,222 UNIV15 4 12,336 UNIV18 9 14,457 UNIV24 11 16,461 UNIV10 4 12,100 UNIV23 9 15,282


********* GROUP 3 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV17 6 12,925 UNIV11 13 17,522 UNIV12 11 16,578 UNIV8 <-- Selected 10 15,754 UNIV3 13 17,404 UNIV9 13 17,404 UNIV22 13 17,522 UNIV25 7 13,396 UNIV13 4 12,218



(Rev. 10/2004) Page 2-31




UNITS SECONDARY PRIMARY INPRIMARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP========================= ============= ============= ============= =====UNIV7 16,696 11 84 9UNIV20 15,400 10 79 9UNIV8 15,754 10 90 9

The selected universities are UNIV7, UNIV8, and UNIV20. A sample of 250 vouchers is

obtained at each university with the following results:

Number of Number of University sampled vouchers vouchers in error UNIV7 250 8 UNIV20 250 12 UNIV8 250 5

This information is recorded in the data file (RHC2DATA.TXT) required by the appraisal

program (TWO-STAGE RHC) and is shown below:

1 250 8 2 250 12 <-- Data file RHC2DATA.TXT 3 250 5

The final portion of the preceding output was stored by the sample selection program in file

RHC2PU.TXT. This file is shown below:

Primary unit file RHC2PU.TXT

UNIV7 16696 11 84 9UNIV20 15400 10 79 9UNIV8 15754 10 90 9

Using these two files, the following output is generated by the TWO-STAGE RHC program:


Page 2-32 (Rev. 10/2004)

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 TWO STAGE RHC ATTRIBUTE APPRAISAL Time: 14:40 AUDIT/REVIEW: Example

NAME OF DATA FILE: C:\TEMP\RHC2DATA.TXT NAME OF PRIMARY UNIT FILE: C:\TEMP\RHC2PU.TXT OUTPUT FILE: C:\TEMP\OutRHC2attr.txt

PRIMARY SAMPLE == ATTRIBUTE == UNIT SIZE SAMPLE TOTAL ======= ====== ============= 1 250 8 2 250 12 3 250 5

TOTALS 750 25

UNITSP.U. SECONDARY PRIMARY INNBR PRIMARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP==== ========================= ============= ============= ============= ===== 1 UNIV7 16,696 11 84 9 2 UNIV20 15,400 10 79 9 3 UNIV8 15,754 10 90 9

TOTALS: 47,850 31 253 27

P.U. SAMPLE SECONDARY SIZES POINT NBR SIZE SAMPLE MEAN UNIVERSE RATIO ESTIMATE ==== ====== ============== ============= ======== ============= 1 250 .03 16,696 7.636 4,079.90 2 250 .05 15,400 7.900 5,839.68 3 250 .02 15,754 9.000 2,835.72

TOTALS: 750 47,850 12,755.30

--- VARIANCE COMPONENTS ---

P.U. WITHIN BETWEEN TOTAL NBR VARIANCE VARIANCE VARIANCE ==== ======== ======== ======== 1 260,846.68 32,187.19 293,033.87 2 338,251.15 4,907,290.25 5,245,541.41 3 173,034.95 3,618,097.64 3,791,132.59

TOTALS: 772,132.78 8,557,575.09 9,329,707.87

PRIMARY UNITS SAMPLED: 3PRIMARY UNITS NOT SAMPLED: 24PRIMARY UNITS IN POPULATION: 27

PROJECTED QUANTITY IN UNIVERSE: 12,755STANDARD ERROR: 3,054


(Rev. 10/2004) Page 2-33

CONFIDENCE LEVEL 80 PERCENT 90 PERCENT 95 PERCENTLOWER LIMIT 8,841 7,731 6,769UPPER LIMIT 16,670 17,779 18,742PRECISION AMOUNT 3,914 5,024 5,987PRECISION PERCENT 30.69% 39.39% 46.93%Z-VALUE USED 1.281551565545 1.644853626951 1.959963984540

Final results: The point estimate of the total number of vouchers in error is 12,755 with a

corresponding standard error of 3,054. The resulting 90% confidence interval is from 7,731 to

17,779 vouchers. Notice that this a very wide confidence interval. The PRECISION PERCENT

is 39.39%, obtained by multiplying the standard error by 1.644853626951 and dividing by the

point estimate (expressed as a percentage); that is (100)(1.644853626951)(3054)/12755. In

general, this can be reduced by sampling a larger number of P.U.s.

Discussion

For this example, = 8/250 = .032, = 12/250 = .048, and = 5/250 = .020. Referring to$p1 $p2 $p3

the Formula section on the next page, the estimate for the total number of vouchers containing

improper charges is:

= (84/11)(16696)(.032) + (79/10)(15400)(.048) + (90/10)(15754)(.02)$T

= 4,079.90 + 5,839.68 + 2,835.72 = 12,755 (rounded)

To determine the variance of , the first component of this variance is the “within variance”$T

equal to V2 = 772,132.78. This accounts for the variation within the primary units (universities).

The larger variance component is the variation between the primary units measured by V1 =

8,557,575.09. The total variance is V1 + V2 = 9,329,707.87 and the estimated standard error of

is = 3,054.$T 9 329 707 87, , .


Page 2-34 (Rev. 10/2004)

The 95% confidence interval for T is 12,755 ± (1.959963984540)(3,054) ; that is 6,769 to

18,742.

FORMULAS

Definitions

1. P.U. stands for primary unit and S.U. is secondary unit

2. Ai = size of i-th P.U.

3. Si = (size of i-th P.U.)/(size of entire population) = Ai/(size of entire population)

4. Bi = total size for i-th group

5. Bi = (total size for i-th group)/(size of entire population) = Bi/(size of entire population)

6. N = number of P.U.s in the population

7. Ni = number of P.U.s in the i-th group

8. n = number of P.U.s in the sample

9. Mi = number of S.U.s in the i-th sampled P.U. (population)

10. mi = number of S.U.s in the i-th sampled P.U. (sample)

Estimator of population total (T)

$ $T BA

M pi

ii

n

i i=⎛⎝⎜

⎞⎠⎟

=∑

1

where = proportion of mi sampled S.U.s having the attribute of interest.$pi

Estimated variance of $T

= V1 + V2 wherev T( $)


(Rev. 10/2004) Page 2-35

VN N

N N

M pS

Ti

i

n

ii

n ii i

ii

n

1

2

1

2 2

1

1

2

=−

−−

⎛⎝⎜

⎞⎠⎟=

=

=

∑

∑∑π

$ $

and

( )V MS

M m p pmi

i

ii

n

i ii i

i2

1

11

= −−−=

∑π$ ( $ )

NOTE: The estimated standard error of is .$T v T( $)

Approximate 95% confidence interval for the population total (T)

$ . ( $)T v T±1959963984540

NOTE: For a 90% confidence interval, replace 1.959963984540 with 1.644853626951 andfor an 80% confidence interval, replace 1.959963984540 with 1.281551565545.

RHC Three Stage Attribute Appraisal RAT-STATS Companion Manual

Page 2-36 (Rev. 10/2004)

RHC Three StageThe RHC sampling procedure can used for a three-stage design.

The steps for such a procedure are the following:

1. A sample of primary units (clusters) is obtained as in the one- and two-stage procedures. The size of the primary units is considered for this sample, where pps sampling is usedfor each group of primary units.

2. A sample of secondary units is obtained within each chosen primary unit by partitioningthe primary unit into random groups. The group sizes are chosen to be as nearly equal aspossible. Using pps sampling, and the size of each secondary unit, one secondary unit ischosen from each of the secondary groups.

3. A random sample of third-stage units is obtained for each of the chosen secondary units. No attention is paid to “size” here. This is a random sample.

Prior to running the appraisal program, the user must run the RAT-STATS RHC SAMPLE

SELECTION program.

Example 7. The situation discussed in Example 5 in the THREE-STAGE UNRESTRICTED

section will be appraised using the RHC methodology. For this example, the stages are:






(universities) in this region and (2) the size of this region (total dollars of grants). This file is

GRANTSPU.TXT. The selected regions are 4, 6, 8, and 10 and the output file created by the

RHC SAMPLE SELECTION program is GRANTSPUOUT.TXT.

RAT-STATS Companion Manual RHC Three Stage Attribute Appraisal

(Rev. 10/2004) Page 2-37

Data set GRANTSPU.TXT

(1) (2) (3)REGION1 117 1250REGION2 63 610REGION3 91 720REGION4 123 1320REGION5 107 1160REGION6 116 1240REGION7 102 960REGION8 118 1300REGION9 122 1320REGION10 85 640REGION11 94 930REGION12 62 550

NOTE: It is okay to set the number of S.U.s [column (2)]equal to one in this file. The actual number ofS.U.s must be known for the selected P.U.s. Thecorrect number of S.U.s must then be insertedinto file GRANTSPUOUT.TXT (the highlightedvalues).

Columns: (1) unit ID(2) number of universities (S.U.s)(3) size (total grant dollar amount x $100,000)

--- Data set GRANTSPUOUT.TXT ---

REGION6 116 1240 3100 3REGION4 123 1320 3410 3REGION8 118 1300 3170 3REGION10 85 640 2320 3


Page 2-38 (Rev. 10/2004)

In practice, do not set these seed values.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 GENERATION OF PRIMARY UNIT SAMPLE Time: 15:09 NAME OF INPUT FILE: C:\TEMP\GRANTSPU.TXT


********* GROUP 1 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= REGION2 610 63 REGION6 <-- Selected 1,240 116 REGION1 1,250 117

GROUP TOTALS: 3 3,100 296

********* GROUP 2 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= REGION4 <-- Selected 1,320 123 REGION5 1,160 107 REGION11 930 94




********* GROUP 4 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= REGION3 720 91 REGION7 960 102 REGION10 <-- Selected 640 85





(Rev. 10/2004) Page 2-39

< Program output - continued > UNITS SECONDARY PRIMARY INPRIMARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP========================= ============= ============= ============= =====REGION6 116 1,240 3,100 3REGION4 123 1,320 3,410 3REGION8 118 1,300 3,170 3REGION10 85 640 2,320 3

NOTE: This is file GRANTSPUOUT.TXT


The input for three-stage RHC program can be greatly simplified if you only obtain information

for each selected primary unit (i.e., regions 4, 6, 8, and 10 here). The information consists of the

size of each secondary unit (here, university) and the number of third-stage units in the universe

for each secondary unit (it is acceptable to set these equal to one and change later). This input is

shown in files REGION4.TXT, REGION6.TXT, REGION8.TXT, and REGION10.TXT. Each

line in the files contains the number of third-stage units (grants) in the universe and the size of

that secondary unit (total grant $ x $100,000), in that order.

After each of these four files is the computer output using the RHC SAMPLE SELECTION

program. A sample of 10 universities is selected for each region. The results are:

REGION UNIVERSITIES 4 85, 46, 7, 82, 30, 34, 27, 66, 65, 80 6 113, 43, 78, 104, 89, 112, 30, 65, 3, 99 8 112, 6, 77, 93, 75, 111, 62, 115, 70, 99 10 78, 43, 7, 73, 55, 33, 10, 59, 64, 39

The previous five program runs (one at the primary level and four at the secondary level) created

five output files. Using a word processor or spreadsheet, these files can be joined to form one of

the input files (the one containing primary/secondary unit information) for the three-stage RHC

program which calculates the confidence interval. The file for this example is

PUSURHC3.TXT.


Page 2-40 (Rev. 10/2004)

Data set REGION4.TXT < - - - continued - - - > < - - - continued - - - >

(1) (2) (3)UNIV1 52 11UNIV2 37 9UNIV3 38 9UNIV4 20 5UNIV5 69 15UNIV6 69 15UNIV7 77 17UNIV8 32 7UNIV9 49 10UNIV10 73 15UNIV11 21 5UNIV12 62 13UNIV13 55 11UNIV14 59 12UNIV15 55 11UNIV16 36 8UNIV17 51 11UNIV18 26 7UNIV19 25 6UNIV20 73 15UNIV21 71 15UNIV22 47 10UNIV23 34 8UNIV24 25 6UNIV25 39 9UNIV26 49 10UNIV27 76 16UNIV28 21 5UNIV29 33 8UNIV30 54 11UNIV31 45 10UNIV32 74 16UNIV33 69 14UNIV34 50 10UNIV35 29 7UNIV36 56 12UNIV37 64 14UNIV38 66 14UNIV39 63 14UNIV40 57 12UNIV41 71 15UNIV42 45 10UNIV43 21 5UNIV44 46 10UNIV45 48 10UNIV46 44 9UNIV47 71 15UNIV48 67 14UNIV49 23 6UNIV50 54 11

UNIV51 62 13UNIV52 52 11UNIV53 56 11UNIV54 70 15UNIV55 41 9UNIV56 65 14UNIV57 76 16UNIV58 30 7UNIV59 75 16UNIV60 27 7UNIV61 36 8UNIV62 61 13UNIV63 58 12UNIV64 61 13UNIV65 62 14UNIV66 76 16UNIV67 71 15UNIV68 34 8UNIV69 62 13UNIV70 23 6UNIV71 28 7UNIV72 46 10UNIV73 62 14UNIV74 67 14UNIV75 25 6UNIV76 24 6UNIV77 57 12UNIV78 44 10UNIV79 73 16UNIV80 70 15UNIV81 45 10UNIV82 52 11UNIV83 34 8UNIV84 59 12UNIV85 54 11UNIV86 31 7UNIV87 69 14UNIV88 22 6UNIV89 47 10UNIV90 57 12UNIV91 31 7UNIV92 73 15UNIV93 52 11UNIV94 22 6UNIV95 22 6UNIV96 29 7UNIV97 56 12UNIV98 74 16UNIV99 43 9UNIV100 57 12

UNIV101 34 8UNIV102 28 7UNIV103 73 15UNIV104 65 14UNIV105 68 14UNIV106 28 7UNIV107 55 11UNIV108 37 9UNIV109 54 11UNIV110 47 10UNIV111 44 9UNIV112 24 6UNIV113 50 10UNIV114 52 11UNIV115 66 14UNIV116 50 10UNIV117 66 14UNIV118 34 8UNIV119 73 16UNIV120 37 8UNIV121 42 9UNIV122 59 12UNIV123 45 11

NOTE: This file has123 lines.

Columns: (1) unit ID (2) number of grants (3) size of university (grant amount x $100,000)


(Rev. 10/2004) Page 2-41

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/25/2004 GENERATION OF SECONDARY UNIT SAMPLE Time: 14:21 NAME OF INPUT FILE: C:\TEMP\REGION4.TXT

GROUPS OF SECONDARY UNITS

********* GROUP 1 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV57 16 76 UNIV48 14 67 UNIV35 7 29 UNIV107 11 55 UNIV85 <-- Selected 11 54 UNIV103 15 73 UNIV86 7 31 UNIV2 9 37 UNIV81 10 45 UNIV58 7 30 UNIV36 12 56 UNIV49 6 23


********* GROUP 2 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV52 11 52 UNIV6 15 69 UNIV46 <-- Selected 9 44 UNIV69 13 62 UNIV108 9 37 UNIV44 10 46 UNIV50 11 54 UNIV121 9 42 UNIV1 11 52 UNIV43 5 21 UNIV87 14 69 UNIV39 14 63


< GROUPS 3 THROUGH 9 ARE OMITTED HERE >

********* GROUP 10 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV53 11 56 UNIV24 6 25 UNIV42 10 45 UNIV120 8 37 UNIV105 14 68 UNIV97 12 56 UNIV119 16 73 UNIV32 16 74 UNIV80 <-- Selected 15 70


Page 2-42 (Rev. 10/2004)

UNIV96 7 29 UNIV13 11 55 UNIV62 13 61 UNIV59 16 75


DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/25/2004 GENERATION OF SECONDARY UNIT SAMPLE Time: 14:21 NAME OF OUTPUT FILE: C:\TEMP\OutRegion4.txt


NUMBER OF SECONDARY UNITS IN THE POPULATION: 123 NUMBER OF SECONDARY UNITS SAMPLED: 10

UNITS 3RD STAGE SECONDARY INSECONDARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP========================= ============= ============= ============= =====UNIV85 54 11 125 12UNIV46 44 9 131 12UNIV7 77 17 119 12UNIV82 52 11 129 12UNIV30 54 11 141 12UNIV34 50 10 140 12UNIV27 76 16 138 12UNIV66 76 16 128 13UNIV65 62 14 125 13UNIV80 70 15 155 13


(Rev. 10/2004) Page 2-43

UNIV1 56 10UNIV2 27 5UNIV3 56 11UNIV4 23 5UNIV5 72 13UNIV6 24 5UNIV7 61 11UNIV8 65 12UNIV9 68 13UNIV10 40 8UNIV11 64 12UNIV12 66 13UNIV13 80 14UNIV14 53 9UNIV15 36 7UNIV16 53 10UNIV17 47 9UNIV18 73 14UNIV19 41 8UNIV20 58 11UNIV21 45 9UNIV22 43 8UNIV23 56 10UNIV24 35 7UNIV25 34 7UNIV26 65 13UNIV27 78 14UNIV28 35 7UNIV29 31 6UNIV30 58 11UNIV31 29 6UNIV32 76 14UNIV33 57 10UNIV34 42 8UNIV35 69 13UNIV36 58 11UNIV37 31 6UNIV38 33 6UNIV39 40 8UNIV40 51 9UNIV41 60 11UNIV42 78 14UNIV43 39 7UNIV44 46 9UNIV45 58 11UNIV46 59 11UNIV47 53 10UNIV48 57 10UNIV49 28 6UNIV50 63 12UNIV51 31 6UNIV52 60 11UNIV53 30 6UNIV54 30 6UNIV55 40 8UNIV56 26 5UNIV57 24 5UNIV58 44 8

<-- continued -->UNIV59 67 13UNIV60 56 10UNIV61 33 7UNIV62 40 8UNIV63 68 13UNIV64 70 13UNIV65 57 10UNIV66 40 7UNIV67 54 10UNIV68 65 12UNIV69 62 12UNIV70 28 5UNIV71 56 10UNIV72 41 8UNIV73 31 6UNIV74 31 6UNIV75 46 9UNIV76 38 7UNIV77 62 12UNIV78 63 12UNIV79 50 9UNIV80 53 9UNIV81 39 7UNIV82 39 7UNIV83 39 7UNIV84 25 5UNIV85 67 13UNIV86 47 9UNIV87 54 10UNIV88 50 9UNIV89 35 7 UNIV90 66 13UNIV91 65 12UNIV92 71 13UNIV93 29 6UNIV94 74 14UNIV95 66 13UNIV96 71 13UNIV97 43 8UNIV98 62 11UNIV99 80 14UNIV100 57 11UNIV101 22 5UNIV102 33 6UNIV103 78 5UNIV104 25 9UNIV105 76 8UNIV106 39 5UNIV107 48 5UNIV108 54 7UNIV109 63 12UNIV110 28 8UNIV111 69 8UNIV112 27 10UNIV113 33 8UNIV114 52 7UNIV115 33 15UNIV116 23 10

Data set REGION6.TXT



Page 2-44 (Rev. 10/2004)

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/25/2004 GENERATION OF SECONDARY UNIT SAMPLE Time: 13:57 NAME OF INPUT FILE: C:\TEMP\REGION6.txt


********* GROUP 1 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV52 11 60 UNIV45 11 58 UNIV32 14 76 UNIV86 9 47 UNIV113 <-- Selected 8 33 UNIV85 13 67 UNIV109 12 63 UNIV87 10 54 UNIV2 5 27 UNIV80 9 53 UNIV53 6 30


********* GROUP 2 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV33 10 57 UNIV48 10 57 UNIV6 5 24 UNIV43 <-- Selected 7 39 UNIV68 12 65 UNIV41 11 60 UNIV46 11 59 UNIV1 10 56 UNIV40 9 51 UNIV88 9 50 UNIV36 11 58



********* GROUP 10 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV20 11 58 UNIV22 8 43 UNIV39 8 40 UNIV111 8 69 UNIV100 11 57 UNIV29 6 31 UNIV105 8 76 UNIV79 9 50 UNIV99 <-- Selected 14 80 UNIV13 14 80 UNIV60 10 56


(Rev. 10/2004) Page 2-45

UNIV54 6 30







Page 2-46 (Rev. 10/2004)




UNIV101 24 5UNIV102 26 6UNIV103 40 10UNIV104 77 16UNIV105 27 6UNIV106 65 15UNIV107 61 13UNIV108 36 9UNIV109 26 6UNIV110 38 9UNIV111 84 17UNIV112 75 16UNIV113 26 6UNIV114 45 10UNIV115 59 13UNIV116 59 13UNIV117 57 12UNIV118 58 12



(Rev. 10/2004) Page 2-47





********* GROUP 2 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV47 14 62 UNIV50 16 80 UNIV6 <-- Selected 8 34 UNIV44 15 71 UNIV68 16 75 UNIV42 10 49 UNIV48 11 54 UNIV1 15 72 UNIV41 9 36 UNIV89 6 26 UNIV37 7 31



********* GROUP 10 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV21 6 26 UNIV23 13 58 UNIV40 9 36 UNIV110 9 38 UNIV100 7 31 UNIV30 6 26 UNIV104 16 77 UNIV81 15 66 UNIV99 <-- Selected 13 60 UNIV13 8 33 UNIV60 15 66


Page 2-48 (Rev. 10/2004)

UNIV56 12 55







(Rev. 10/2004) Page 2-49

Data setREGION10.TXT

UNIV1 34 6UNIV2 32 5UNIV3 69 10UNIV4 23 4UNIV5 60 9UNIV6 72 11UNIV7 56 9UNIV8 28 5UNIV9 38 6UNIV10 60 9UNIV11 58 9UNIV12 37 6UNIV13 70 10UNIV14 37 6UNIV15 81 12UNIV16 53 9UNIV17 63 10UNIV18 32 5UNIV19 33 5UNIV20 37 6UNIV21 77 11UNIV22 52 8UNIV23 63 10UNIV24 41 7UNIV25 45 8UNIV26 34 6UNIV27 61 10UNIV28 70 10UNIV29 34 5UNIV30 22 4UNIV31 66 10UNIV32 69 10UNIV33 65 10UNIV34 26 4UNIV35 43 7UNIV36 65 10UNIV37 80 12UNIV38 74 11UNIV39 38 6UNIV40 43 7UNIV41 47 8UNIV42 59 9UNIV43 42 7UNIV44 54 9UNIV45 73 11

<--continued -->

UNIV46 78 12UNIV47 72 11UNIV48 30 5UNIV49 47 8UNIV50 52 8UNIV51 24 4UNIV52 26 4UNIV53 22 4UNIV54 57 9UNIV55 78 12UNIV56 62 10UNIV57 57 9UNIV58 68 10UNIV59 52 8UNIV60 54 9UNIV61 41 7UNIV62 61 10UNIV63 79 12UNIV64 50 8UNIV65 54 9UNIV66 53 9UNIV67 40 7UNIV68 44 7UNIV69 39 7UNIV70 72 11UNIV71 76 11UNIV72 34 5UNIV73 27 5UNIV74 40 7UNIV75 41 7UNIV76 25 4UNIV77 41 7UNIV78 39 7UNIV79 58 9UNIV80 71 11UNIV81 37 6UNIV82 30 5UNIV83 78 12UNIV84 59 9UNIV85 29 5

Note: This file has 85 lines.


Page 2-50 (Rev. 10/2004)



********* GROUP 1 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV44 9 54 UNIV32 10 69 UNIV77 7 41 UNIV78 <-- Selected 7 39 UNIV2 5 32 UNIV50 8 52 UNIV34 4 26 UNIV46 12 78


********* GROUP 2 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV6 11 72 UNIV43 <-- Selected 7 42 UNIV62 10 61 UNIV41 8 47 UNIV1 6 34 UNIV40 7 43 UNIV79 9 58 UNIV36 10 65



********* GROUP 10 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV71 11 76 UNIV9 6 38 UNIV21 11 77 UNIV23 10 63 UNIV39 <-- Selected 6 38 UNIV29 5 34 UNIV72 5 34 UNIV13 10 70 UNIV51 4 24



(Rev. 10/2004) Page 2-51






Page 2-52 (Rev. 10/2004)

--- Data set PUSURHC3.TXT ---

REGION4 123 1320 3410 3 10UNIV85 54 11 125 12UNIV46 44 9 131 12UNIV7 77 17 119 12UNIV82 52 11 129 12UNIV30 54 11 141 12UNIV34 50 10 140 12UNIV27 76 16 138 12UNIV66 76 16 128 13UNIV65 62 14 125 13UNIV80 70 15 155 13REGION6 116 1240 3100 3 10UNIV113 33 8 108 11UNIV43 39 7 105 11UNIV78 63 12 104 11UNIV104 25 9 96 11UNIV89 35 7 124 12UNIV112 27 10 108 12UNIV30 58 11 95 12UNIV65 57 10 109 12UNIV3 56 11 115 12UNIV99 80 14 113 12REGION8 118 1300 3170 3 10UNIV112 75 16 125 11UNIV6 34 8 127 11UNIV7 51 11 120 12UNIV93 54 11 136 12UNIV75 52 11 126 12UNIV111 84 17 134 12UNIV62 64 14 123 12UNIV115 59 13 137 12UNIV70 65 14 143 12UNIV99 60 13 129 12REGION10 85 640 2320 3 10UNIV78 39 7 62 8UNIV43 42 7 68 8UNIV7 56 9 54 8UNIV73 27 5 63 8UNIV55 78 12 70 8UNIV33 65 10 77 9UNIV10 60 9 76 9UNIV59 52 8 71 9UNIV64 50 8 73 9UNIV39 38 6 68 9

NOTE: This is the data file constructed using the RHC SAMPLE SELECTION program toselect the primary units (regions) and, within each selected primary unit, the 10secondary units (universities). The four lines beginning with REGIONx are from theoutput file created during the primary unit selection (GRANTSPUOUT.TXT). Avalue of 10 (the number of selected universities for that region) is added to the end ofeach of these lines. The 10 lines after each REGIONx line consist of the output filecreated when selecting the universities from each region (OUTREGION4.TXT, . . .,OUTREGION10.TXT).


(Rev. 10/2004) Page 2-53


Since approximately 20% of the grants at each university in the sample are to be audited, the

following sample sizes are determined:

Region 4: University Grants in universe Number to be audited UNIV85 54 11 UNIV46 44 9 UNIV7 77 15 UNIV82 52 10 UNIV30 54 11 UNIV34 50 10 UNIV27 76 15 UNIV66 76 15 UNIV65 62 12 UNIV80 70 14 122





Page 2-54 (Rev. 10/2004)

The data file containing the errors for these 438 audited grants is RHC3DATA.TXT. Each line

contains (1) a counter, (2) the number of sampled (audited) secondary units (grants), and (3) the

number of grants containing improper charges (in error).

File RHC3DATA.TXT < - - - continued - - - >

1.1 11 21.2 9 41.3 15 31.4 10 21.5 11 51.6 10 21.7 15 21.8 15 41.9 12 11.10 14 32.1 7 22.2 8 42.3 13 52.4 5 12.5 7 32.6 5 22.7 12 32.8 11 22.9 11 42.10 16 3

3.1 15 13.2 7 33.3 10 23.4 11 43.5 10 33.6 17 63.7 13 03.8 12 13.9 13 23.10 12 34.1 8 54.2 8 14.3 11 54.4 5 34.5 16 34.6 13 44.7 12 04.8 10 24.9 10 34.10 8 3

(*)

Note: This filehas 40 lines.

(*) To illustrate, the fifth university in the fourth sampled P.U. (Region 10) had 16 grants (third-

stage units) audited and three of them contained improper charges.

Finally, the three-stage RHC program is run to generate a confidence interval for the universe

total using input files PUSURHC3.TXT and RHC3DATA.TXT. The output from this program

is shown at the end of this section.


(Rev. 10/2004) Page 2-55

Summary of results

The estimate for the number of grants in error for the universe (all 12 regions) is the OVERALL

POINT ESTIMATE of 15,861 with a corresponding estimated OVERALL STANDARD

ERROR of 2,039 grants.

NOTE: This estimate does not require knowing the number of grants in the universe. If thisvalue is known, you can convert the point estimate into a proportion. For example, ifthe total number of grants in all 12 regions is 59,200, then the point estimate for theproportion of grants in error is 15,861/59,200 = .268 (26.8%) with a correspondingstandard error of 2,039/59,200 = .034 (3.4%).

The program also provides estimates for the number of grants in error for each sampled P.U.

(region) and for each of the groups of S.U.s (universities) within each sampled region. For

example, the estimated number of grants in error for Region 4 is 1,531 and the estimated number

of grants in error for the group of nine universities containing UNIV85 is 112. The SIZES

RATIO refers to the ratio of the size of the group containing this university to the size of this

university. To illustrate, UNIV85 in Region 4 has a size of 11 and is in a group of size 125 (look

at file REGION4.TXT). The SIZES RATIO here is 125/11 = 11.3636.

The 95% confidence interval for the number of grants in error is 11,865 to 19,857. If the total

number of third-stage units in the universe is known (say, 59,200), this interval can be converted

into an interval for the proportion of grants in error by dividing both limits by this value (here,

.200 to .335).

The PRECISION AMOUNT is the amount added and subtracted to the point estimate (15,861)

to obtain the corresponding confidence interval. In the 95% confidence interval, the lower limit

of 11,865 is obtained by subtracting the precision amount of 3,996 from 15,861. The


Page 2-56 (Rev. 10/2004)

PRECISION PERCENT is the precision amount divided by the point estimate, expressed as a

percentage.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/26/2004 THREE STAGE RHC ATTRIBUTE APPRAISAL Time: 10:04 AUDIT/REVIEW: RHC 3-Stage

DATA FILE USED: C:\TEMP\RHC3DATA.txt PRIMARY/SECONDARY UNIVERSE FILE USED: C:\TEMP\PUSURHC3.txt OUTPUT FILE: C:\TEMP\OutRHC3.txt

*** ATTRIBUTE *** **** SAMPLED UNITS **** THIRD STAGE SAMPLE NO. WITHPRIMARY / SECONDARY IDENTIFICATION UNIVERSE SIZE ATTRIBUTE================================== =========== ====== ============REGION4 UNIV85 54 11 2 UNIV46 44 9 4 UNIV7 77 15 3 UNIV82 52 10 2 UNIV30 54 11 5 UNIV34 50 10 2 UNIV27 76 15 2 UNIV66 76 15 4 UNIV65 62 12 1 UNIV80 70 14 3REGION6 UNIV113 33 7 2 UNIV43 39 8 4 UNIV78 63 13 5 UNIV104 25 5 1 UNIV89 35 7 3 UNIV112 27 5 2 UNIV30 58 12 3 UNIV65 57 11 2 UNIV3 56 11 4 UNIV99 80 16 3REGION8 UNIV112 75 15 1 UNIV6 34 7 3 UNIV7 51 10 2 UNIV93 54 11 4 UNIV75 52 10 3 UNIV111 84 17 6 UNIV62 64 13 0 UNIV115 59 12 1 UNIV70 65 13 2 UNIV99 60 12 3REGION10 UNIV78 39 8 5 UNIV43 42 8 1 UNIV7 56 11 5 UNIV73 27 5 3 UNIV55 78 16 3 UNIV33 65 13 4 UNIV10 60 12 0 UNIV59 52 10 2


(Rev. 10/2004) Page 2-57

UNIV64 50 10 3 UNIV39 38 8 3

TOTALS 2,270 438 111

--- POINT ESTIMATES --- *** ATTRIBUTE *** **** SAMPLED UNITS **** SIZES POINT PRIMARY / SECONDARY IDENTIFICATION SAMPLE MEAN RATIO ESTIMATE ================================== =========== =========== =============REGION4 UNIV85 NOTE: 112 is the estimate 0.18 11.3636 112 UNIV46 for the group containing 0.44 14.5556 285 UNIV7 UNIV85 (not just UNIV85). 0.20 7.0000 108 UNIV82 This group contained 12 0.20 11.7273 122 UNIV30 universities shown earlier 0.45 12.8182 315 UNIV34 in the output using 0.20 14.0000 140 UNIV27 data set REGION4.TXT. 0.13 8.6250 87 UNIV66 0.27 8.0000 162 UNIV65 0.08 8.9286 46 UNIV80 0.21 10.3333 155

TOTAL Estimate for Region 4 º 1,531

REGION6 UNIV113 0.29 13.5000 127 UNIV43 0.50 15.0000 293 UNIV78 0.38 8.6667 210 UNIV104 0.20 10.6667 53 UNIV89 0.43 17.7143 266 UNIV112 0.40 10.8000 117 UNIV30 0.25 8.6364 125 UNIV65 0.18 10.9000 113 UNIV3 0.36 10.4545 213 UNIV99 0.19 8.0714 121

TOTAL 1,638


TOTAL 1,340

REGION10 UNIV78 0.63 8.8571 216 UNIV43 0.13 9.7143 51 UNIV7 0.45 6.0000 153 UNIV73 0.60 12.6000 204 UNIV55 0.19 5.8333 85 UNIV33 0.31 7.7000 154


Page 2-58 (Rev. 10/2004)

UNIV10 0.00 8.4444 0 UNIV59 0.20 8.8750 92 UNIV64 0.30 9.1250 137 UNIV39 0.38 11.3333 162

TOTAL 1,254

--- VARIANCE COMPONENTS FOR PRIMARY UNITS ---

**** SAMPLED UNITS **** WITHIN BETWEEN TOTALPRIMARY UNIT IDENTIFICATION VARIANCE VARIANCE VARIANCE============================== ============== ============== ==============REGION4 4,722 59,965 64,687REGION6 4,734 59,651 64,385REGION8 4,102 68,169 72,272REGION10 3,351 47,934 51,285 (Values of V4) (Values of V3)

--- COMBINED VARIANCE COMPONENTS ---

STAGE 1 STAGES 2 AND 3 TOTAL VARIANCE======================= ======================= =======================

3,466,892 690,212 4,157,104 (Value of V1) (Value of V2)

*** ATTRIBUTE *** --- SUMMARY OF APPRAISAL RESULTS ---PRIMARY UNITS SAMPLED 4PRIMARY UNITS NOT SAMPLED 8TOTAL PRIMARY UNITS 12

PROJECTED QUANTITY IN UNIVERSE 15,861STANDARD ERROR 2,039



(Rev. 10/2004) Page 2-59

FORMULAS

Definitions

1. Si = (size of i-th P.U.)/(size of entire population)

2. Bi = ESi over the i-th group of P.U.s

3. Sij = (size of j-th S.U. in the i-th sampled P.U.)/(size of i-th sampled P.U.)

(Note: denominator of Sij = numerator of Si)

4. Bij = ESij over the j-th group in i-th sampled P.U.

5. N = number of P.U.s (population)

6. n = number of P.U.s (sample)

7. Mi = number of S.U.s in i-th sampled P.U. (population)

8. mi = number of S.U.s in i-th sampled P.U. (sample)

9. Kij = number of records for j-th sampled S.U. in i-th sampled P.U. (population)

10. kij = number of records for j-th sampled S.U. in i-th sampled P.U. (sample)


$$

TTSi

i

ii

n

=⎛

⎝⎜

⎞

⎠⎟

=∑π

1

where = estimator of total for i-th sampled P.U.$Ti

(equation 1)π ijj

mij

ij

i TS=

∑⎛

⎝⎜⎜

⎞

⎠⎟⎟

1

$

and = estimator of population total for j-th sampled S.U. in i-th sampled P.U.$Tij

= K pij ij$

with = proportion of kij records having the attribute of interest$pij


Page 2-60 (Rev. 10/2004)

NOTE: It can be shown that is an unbiased estimator of T.$T



(equation 2)VN N

N N

TS

Ti

i

n

ii

n ii

ni

i1

2

1

2 2

1

1

2

=−

−−

⎛

⎝⎜

⎞

⎠⎟=

=

=

∑

∑∑π

$$

and

(equation 3)VS

v Ti

ii

n

i21

=⎛⎝⎜

⎞⎠⎟

=∑ π

( $ )

and Ni = number of P.U.s in the i-th group after the random split into n groups. is obtained by applying the two stage RHC procedure within the i-th sampled P.U.; i.e.,v Ti( $ )

the i-th P.U. is viewed as the entire population. Consequently, ,wherev T V Vi i i( $ ) , ,= +3 4

VM M

M M

K pS

Ti

ij ij

m

i ijj

m ijj

mij ij

iji

i

i

i

3

2

1

2 2

1

1

2

,

$$=

−

−−

⎛

⎝⎜⎜

⎞

⎠⎟⎟

=

=

=

∑

∑∑π

and

VKS

K kp p

ki ijij

ijj

m

ij ijij ij

ij

i

41

11, ( )

$ ( $ )= −

−

−=∑π

and where (1) Mij = the number of S.U.s in the j-th random group, i-th sampled P.U.

(2) = proportion of the kij items having the attribute of interest for the j-th$pij

sampled S.U. within the i-th sampled P.U.


(Rev. 10/2004) Page 2-61

Comments 1. V1 is essentially the same expression obtained for the single-stage RHC

procedure and will be referred to as the “between unit” variation.

2. V2 is the contribution of the second- and third-stage variation and is obtained by

treating each sampled P.U. as the population to be sampled using two

(additional) stages.

3. The estimated standard error of is .$T v T( $)


$ . ( $)T v T±1959963984540

NOTE: For a 90% confidence interval, replace 1.959963984540 with 1.644853626951 and for an 80% confidence interval, replace 1.959963984540 with 1.281551565545.

Stratified Cluster Attribute Appraisal RAT-STATS Companion Manual

Page 2-62 (Rev. 10/2004)

Stratified Cluster

With this procedure, you first stratify, then obtain a cluster (single-stage) sample within each

stratum. This is motivated by the discussion in the RAT-STATS User’s Guide.

Example 8. In a large section of the U.S., an audit was conducted for 583 universities with

health-related research grants. Two strata were defined:

Stratum 1: state-supported universities and

Stratum 2: private universities.

It was decided to estimate the proportion of contracts containing charges after the scheduled

completion of the contract using the same two strata. The strata sizes were N1 = 415 and N2 =

168. Within each stratum, a single-stage cluster sample was obtained with n1 = 25 universities

selected from Stratum 1 (state supported universities) and n2 = 10 universities from Stratum 2

(private universities). The total number of grants in the universe for Stratum 1 is 2,500 and for

Stratum 2, the total number is 1,000 grants. Consequently, there are 3,500 grants in the entire

universe.

NOTE 1: These sample sizes are not adequate according to OAS policy and are used here forillustration purposes only.

NOTE 2: This procedure does require knowledge of the total number of elements in theuniverse for each stratum.

The following data were obtained, where aj is the number of grants containing charges after the

scheduled completion of the grant for the j-th university, Mj is the number of grants (universe)

for this university (all of which are audited), and pj is aj/Mj.

RAT-STATS Companion Manual Stratified Cluster Attribute Appraisal

(Rev. 10/2004) Page 2-63

Summary Using Computer Output and Corresponding Formulas:

Stratum 1: Eaj = 38, EMj = 151, = 38/151 = .2517$p1

The projected number in the universe for Stratum 1 is

= (2500)(.2517) = 629$T1

Stratum 2: Eaj = 19, EMj = 49, = 19/49 = .3878$p2

The projected number in the universe for Stratum 2 is

= (1000)(.3878) = 388$T2

The estimate of the total number of grants in the universe with charges after the

scheduled grant completion is

= 629 + 388 = 1017$T

The estimated proportion of grants with such charges is

= 1017 / 3500 = .2905$p

The estimated standard error for (using the square root of Equation 5) is 46.$T

The estimated standard error for (using the square root of Equation 6) is .0132.$p


Page 2-64 (Rev. 10/2004)

Stratum 1 -- State Universities

Univ. Mj aj pj Univ. Mj aj pj 1 8 2 .25 14 10 3 .30 2 12 3 .25 15 9 1 .11 3 4 2 .50 16 3 1 .33 4 5 1 .20 17 6 2 .33 5 6 1 .17 18 5 1 .20 6 6 2 .33 19 5 1 .20 7 7 2 .29 20 4 1 .25 8 5 2 .40 21 6 1 .17 9 8 2 .25 22 8 1 .12 10 3 1 .33 23 7 2 .29 11 2 0 .00 24 3 1 .33 12 6 2 .33 25 8 2 .25 13 5 1 .20

Stratum 2 -- Private Universities

Univ. Mj aj pj Univ. Mj aj pj 1 2 1 .50 6 8 3 .37 2 5 2 .40 7 6 2 .33 3 7 2 .29 8 10 4 .40 4 4 2 .50 9 3 1 .33 5 3 1 .33 10 1 1 1.00

These results are combined into data set DATACLUS.TXT (39 lines)

Data set

STATE UNIVERSITIES 415 25 2500UNIV1 8 2UNIV2 12 3UNIV3 4 2UNIV4 5 1UNIV5 6 1UNIV6 6 2UNIV7 7 2UNIV8 5 2UNIV9 8 2UNIV10 3 1UNIV11 2 0UNIV12 6 2UNIV13 5 1UNIV14 10 3UNIV15 9 1UNIV16 3 1UNIV17 6 2


(Rev. 10/2004) Page 2-65

UNIV18 5 1UNIV19 5 1UNIV20 4 1UNIV21 6 1UNIV22 8 1UNIV23 7 2UNIV24 3 1UNIV25 8 2PRIVATE UNIVERSITIES 168 10 1000UNIV1 2 1UNIV2 5 2UNIV3 7 2UNIV4 4 2UNIV5 3 1UNIV6 8 3UNIV7 6 2UNIV8 10 4UNIV9 3 1UNIV10 1 1

The following computer printout is produced:

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 2/16/2004 STRATIFIED CLUSTER ATTRIBUTE APPRAISAL Time: 15:36 AUDIT/REVIEW: Attribute - Stratified Cluster NAME OF INPUT FILE: C:\Temp\DATACLUS.TXT

STRATUM IDENTIFICATION SAMPLE SAMPLE MEETING PROJECTED CLUSTER IDENTIFICATION UNIVERSE SIZE CRITERIA PERCENT QUANTITY=========================== =========== ====== ======== ======= =========STATE UNIVERSITIES 415 25 UNIV1 8 8 2 UNIV2 12 12 3 UNIV3 4 4 2 UNIV4 5 5 1 UNIV5 6 6 1 UNIV6 6 6 2 UNIV7 7 7 2 UNIV8 5 5 2 UNIV9 8 8 2 UNIV10 3 3 1 UNIV11 2 2 0 UNIV12 6 6 2 UNIV13 5 5 1 UNIV14 10 10 3 UNIV15 9 9 1 UNIV16 3 3 1 UNIV17 6 6 2 UNIV18 5 5 1 UNIV19 5 5 1 UNIV20 4 4 1 UNIV21 6 6 1 UNIV22 8 8 1


Page 2-66 (Rev. 10/2004)

UNIV23 7 7 2 UNIV24 3 3 1 UNIV25 8 8 2

STRATUM TOTALS 2,500 151 38 25.17% 629

PRIVATE UNIVERSITIES 168 10 UNIV1 2 2 1 UNIV2 5 5 2 UNIV3 7 7 2 UNIV4 4 4 2 UNIV5 3 3 1 UNIV6 8 8 3 UNIV7 6 6 2 UNIV8 10 10 4 UNIV9 3 3 1 UNIV10 1 1 1

STRATUM TOTALS 1,000 49 19 38.78% 388

STRATA TOTALS 583 35CLUSTER UNIT TOTALS 3,500 200 57OVERALL TOTALS 29.05% 1,017OVERALL STANDARD ERROR 1.32% 46


LOWER LIMIT FOR TOTAL 958 941 926UPPER LIMIT FOR TOTAL 1,076 1,093 1,107

FORMULAS

1. Estimated proportion in stratum h that possess the attribute of interest

$

,

,

p

a

M

h

j hj

n

j hj

n

h

h=

=

=

∑

∑1

1

where aj, h is the number of elements in the j-th secondary unit in stratum h possessing the

attribute of interest, Mj,h is the number of secondary units in the j-th primary unit in stratum

h, and nh is the number of sample items in stratum h.


(Rev. 10/2004) Page 2-67

2. Estimated total number of elements in stratum h that possess the attribute of interest

$ $T M ph h h=

where Mh = number of secondary units in the universe for stratum h (must be known)

3. Estimated universe total having the attribute of interest

summed over the L strata$ $T Thh

L

==∑

1

4. Estimated universe proportion having the attribute of interest is where M is the$ $ /p T M=

total number of secondary units in the universe and M = EMh (summed over the L strata).

5. Estimated variance of $T

v TN N n

n na p Mh h h

h hh

L

j h h j hj

nh

( $)( )( )

( $ ), ,=−−

−= =∑ ∑1

1

2

1

where Nh is the number of universe items in stratum h.

6. Estimated variance of $p

v p v T M( $) ( $) /= 2

7. Approximate 95% confidence interval for T

$ $T ± 1.959963984540 v(T)

8. Approximate 95% confidence interval for p

$ $p ± 1.959963984540 v(p)

NOTE: For the Precision at the 90% Confidence Level, replace 1.959963984540 with1.644853626951 and for the Precision at the 80% Confidence Level,replace1.959963984540 with 1.281551565545.

Stratified Multistage Attribute Appraisal RAT-STATS Companion Manual

Page 2-68 (Rev. 10/2004)

Stratified Multistage

As with the stratified cluster procedure, you must first stratify the universe. Rather than take a

cluster (single-stage) sample within each stratum, you will obtain a multistage (two-stage or

three-stage) sample within each stratum. These multistage samples may be random (using the

Two-Stage Unrestricted or Three-Stage Unrestricted programs) or may be obtained using the

RHC procedure and the RHC Two-Stage or RHC Three-Stage programs.

Unlike the Stratified Cluster program, this program requires that you first run the appropriate

multistage program on each stratum and record the results. The output results are then used as

input to the Stratified Multistage program. You may store the results from each stratum (point

estimate, standard error, universe size) in a file or simply input these values interactively.

NOTE: The “universe size” refers to the number of units at the most detailed level of themultistage sample. For example, if you are obtaining a three-stage sample withineach stratum, then the “universe size” refers to the total number of third-stage unitswithin this stratum.

Example 9. This example is similar to Example 8 in the Stratified Cluster section. The universe

consisting of university grants is stratified by defining

Stratum 1: state-supported universities (5,600 grants) and

and Stratum 2: private universities (3,500 grants)

Because these universities are so widespread, it was decided to employ a two-stage sample using

20 state-supported universities and 10 private universities. Rather than audit all grants at a

selected university, it was decided (based on available resources) to audit roughly 20% of the

grants at each selected university to estimate the proportion of grants containing charges after the

RAT-STATS Companion Manual Stratified Multistage Attribute Appraisal

(Rev. 10/2004) Page 2-69

scheduled completion of the grant. The following data were obtained, where ai is the number of

grants in the sample from the i-th university containing such charges, mi is the number of audited

(sampled) grants at the i-th university, and Mi is the total number of grants in the audit universe

at the i-th university.

State-supported universities Private universities

Univ. Mi mi ai Univ. Mi mi ai 1 41 8 2 1 11 2 1 2 62 12 3 2 25 5 2 3 21 4 2 3 34 7 2 4 23 5 1 4 18 4 2 5 31 6 1 5 16 3 1 6 32 6 2 6 40 8 3 7 33 7 2 7 31 6 2 8 27 5 2 8 50 10 4 9 41 8 2 9 14 3 1 10 16 3 1 10 12 2 1 11 9 2 0 These values are stored in data set MULSTAT2.TXT. 12 31 6 2 13 27 5 1 14 49 10 3 15 46 9 1 16 15 3 1 17 30 6 2 18 24 5 1 19 23 5 1 20 21 4 1These values are stored in data set MULSTAT1.TXT.


Page 2-70 (Rev. 10/2004)

The following two computer outputs are obtained using the Two-Stage Unrestricted program.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 2/11/2004 TWO STAGE UNRESTRICTED ATTRIBUTE APPRAISAL Time: 15:02 AUDIT/REVIEW: Stratum1 DATA FILE: C:\Temp\MULSTAT1.TXT


1 41 8 2 25.00% 10 2 62 12 3 25.00% 16 3 21 4 2 50.00% 11 4 23 5 1 20.00% 5 5 31 6 1 16.67% 5 6 32 6 2 33.33% 11 7 33 7 2 28.57% 9 8 27 5 2 40.00% 11 9 41 8 2 25.00% 10 10 16 3 1 33.33% 5 11 9 2 0 0.00% 0 12 31 6 2 33.33% 10 13 27 5 1 20.00% 5 14 49 10 3 30.00% 15 15 46 9 1 11.11% 5 16 15 3 1 33.33% 5 17 30 6 2 33.33% 10 18 24 5 1 20.00% 5 19 23 5 1 20.00% 5 20 21 4 1 25.00% 5

TOTALS 602 119 31

OVERALL TOTALS 120 5,600 16.90% 946





(Rev. 10/2004) Page 2-71

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 2/11/2004 TWO STAGE UNRESTRICTED ATTRIBUTE APPRAISAL Time: 15:13 AUDIT/REVIEW: Stratum 2 DATA FILE: C:\Temp\MULSTAT2.TXT


1 11 2 1 50.00% 6 2 25 5 2 40.00% 10 3 34 7 2 28.57% 10 4 18 4 2 50.00% 9 5 16 3 1 33.33% 5 6 40 8 3 37.50% 15 7 31 6 2 33.33% 10 8 50 10 4 40.00% 20 9 14 3 1 33.33% 5 10 12 2 1 50.00% 6

TOTALS 251 50 19

OVERALL TOTALS 80 3,500 21.84% 764



LOWER LIMIT FOR TOTAL 604 559 520UPPER LIMIT FOR TOTAL 924 970 1,009

The values used as input to the Stratified Multistage program are highlighted in the preceding

computer output. The following computer screen illustrates how to enter these values:


Page 2-72 (Rev. 10/2004)

The following output is obtained from the Stratified Multistage program.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 2/11/2004 STRATIFIED MULTISTAGE ATTRIBUTE APPRAISAL Time: 16:08 AUDIT/REVIEW: Illustration

THE ESTIMATORS ARE BASED ON THE FOLLOWING ENTRIES: STRATUM ATTRIBUTE STANDARD ERROR UNIVERSE SIZE 1 16.90% 1.99% 5,600 2 21.84% 3.57% 3,500

= = = = = = = = = = = = = = = = = RESULTS = = = = = = = = = = = = = = = = = =

ESTIMATED PERCENTAGE: 18.80%ESTIMATED TOTAL: 1,711

STANDARD ERROR (PERCENTAGE): 1.84%STANDARD ERROR (TOTAL): 167




(Rev. 10/2004) Page 2-73

Final results: The point estimate for the percentage of grants containing improper charges is

18.8% (standard error of 1.84%) and the 90% confidence interval for this proportion is from

15.77% to 21.83%. The 90% confidence interval for the universe total is from 1,435 to 1,986.

Discussion. Using Equation 1 in the Formulas section,

= (5600/9100)(.169) + (3500/9100)(.2184) = .188 (18.8%)$p

and

= (5600/9100)2(.0199)2 + (3500/9100)2(.0357)2 = .000339v p( $)

The estimated standard error of is = .0184 (1.84%).$p v p( $)

The corresponding 90% confidence interval is

18.8 ± 1.644853626951(1.84); that is 15.77% to 21.83%

The estimate of the universe total and corresponding confidence interval are obtained by

multiplying the previous results by the total universe size = 5,600 + 3,500 = 9,100.

FORMULAS

1. Estimated universe proportion having the attribute of interest

$ $pMM

pi

i

L

i=⎛⎝⎜

⎞⎠⎟

=∑

1

where L = number of strata

Mi = universe size for the most detailed level of the multistage sample

M = total universe size = Emi


Page 2-74 (Rev. 10/2004)

= estimated proportion for the i-th stratum$pi

2. Estimated variance of $p

(standard error of )2v pMM

i

i

L

( $) = ⎛⎝⎜

⎞⎠⎟

=∑

1

2

$pi

3. 90% confidence interval for p

$ $p ± 1.644853626951 v(p)

4. Estimated universe total having the attribute of interest

$ $T Mp=


v T M v p( $) ( $)= 2

6. 90% confidence interval for T

$ $T ±1.644853626951 v(T)

NOTE: For the Precision at the 95% Confidence Level, replace 1.644853626951 with1.959963984540 and for the Precision at the 80% Confidence Level, replace1.644853626951 with 1.281551565545.

RAT-STATS Companion Manual Variable Appraisals

(Rev. 10/2004) Page 3-1

# Unrestricted

# Stratified

# Two-Stage Unrestricted

# Three-Stage Unrestricted

# RHC Two Stage

# RHC Three Stage

# Stratified Cluster

# Stratified Multistage

# Post Stratification

# Unknown Universe Size

VARIABLE APPRAISALS

A variable appraisal is carried out to estimate a particular universe total (T) and its

corresponding sampling error. For example, the audit intent may be to determine the dollar

value of an inventory or the amount of duplicate payments made by an organization.

A variety of procedures can be used to obtain and appraise a variable sample. There are ten

sampling strategies utilized in the Variables Appraisals modules. They are listed below and

described in the sections to follow.

Unrestricted Variable Appraisal RAT-STATS Companion Manual

Page 3-2 (Rev. 10/2004)

Unrestricted Variable Appraisal

An unrestricted sample is the same as a simple random sample. Consequently, every sample

of size n has the same chance of being selected. For an unrestricted sample, a sample of size n is

randomly obtained and the variable of interest is recorded for each sample item. Actually, the

user may input a set of single values (examined amounts, audit amounts, or difference amounts)

or a set of two values (examined and audit amounts, examined and difference amounts, or audit

and difference amounts).

Example 1. An unrestricted sample of 50 items resulted in the 50 examined/audited values

contained in data set DATASRS.TXT. For this sample, all of the resulting differences

(examined value - audited value) were nonzero since all the examined (book) values were

unequal to the corresponding audit (actual) values.

Data file DATASRS.TXT

1 300 267 µ Each line contains a line counter, examined value, and audited value 2 900 774 separated by one or more spaces, a tab delimiter, or a comma.3 300 2554 200 1745 900 8106 700 5607 1000 8208 100 809 900 76510 700 63011 700 63012 400 33213 300 25514 100 8415 200 16816 100 8817 600 52818 400 340

RAT-STATS Companion Manual Unrestricted Variable Appraisal

(Rev. 10/2004) Page 3-3

19 900 74720 1000 80021 1000 86222 600 50423 800 64824 200 17625 200 17226 1000 89027 900 79228 600 54029 500 52530 200 17231 200 17832 500 42533 200 16434 500 42035 500 40036 400 32437 200 16038 600 54039 500 42540 300 26441 900 76542 100 8443 100 8544 900 81045 300 24046 500 41547 500 42548 300 23749 500 43550 100 86

The output on the next page was obtained from the Unrestricted Variable Appraisal program.


Page 3-4 (Rev. 10/2004)

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 4/5/2004 VARIABLE UNRESTRICTED APPRAISAL Time: 11:23 AUDIT/REVIEW: Variable SRS DATA FILE USED: C:\Temp\DATASRS.TXT

SAMPLE EXAMINED NONZERO TOTAL OF TOTAL OF SIZE VALUE DIFFS DIFF VALUES AUD VALUES 50 24,800.00 50 3,530.00 21,270.00

----------------------- E X A M I N E D ------------------------ MEAN / UNIVERSE 496.00 10,000 STANDARD DEVIATION 296.90 SKEWNESS .32 KURTOSIS 1.81 STANDARD ERROR (MEAN) 41.88 STANDARD ERROR (TOTAL) 418,823 POINT ESTIMATE 4,960,000

CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT 4,415,921 UPPER LIMIT 5,504,079 PRECISION AMOUNT 544,079 PRECISION PERCENT 10.97% T-VALUE USED 1.299068784748

90% CONFIDENCE LEVEL LOWER LIMIT 4,257,823 UPPER LIMIT 5,662,177 PRECISION AMOUNT 702,177 PRECISION PERCENT 14.16% T-VALUE USED 1.676550892617


----------------------- A U D I T E D -------------------------- MEAN / UNIVERSE 425.40 10,000 STANDARD DEVIATION 256.20 SKEWNESS .30 KURTOSIS 1.78 STANDARD ERROR (MEAN) 36.14 STANDARD ERROR (TOTAL) 361,412 POINT ESTIMATE 4,254,000



(Rev. 10/2004) Page 3-5



--------------------- D I F F E R E N C E ---------------------- MEAN / UNIVERSE 70.60 10,000 STANDARD DEVIATION 48.25 SKEWNESS .64 KURTOSIS 2.98 STANDARD ERROR (MEAN) 6.81 STANDARD ERROR (TOTAL) 68,068 POINT ESTIMATE 706,000

CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT 617,575 UPPER LIMIT 794,425 PRECISION AMOUNT 88,425 PRECISION PERCENT 12.52% T-VALUE USED 1.299068784748

90% CONFIDENCE LEVEL LOWER LIMIT 591,881 UPPER LIMIT 820,119 PRECISION AMOUNT 114,119 PRECISION PERCENT 16.16% T-VALUE USED 1.676550892617



1 When the POINT ESTIMATE is negative, the PRECISION PERCENT is set equal to zero.

Page 3-6 (Rev. 10/2004)

Explanation.

NOTE: The following discussion can be applied to the examined values, the audited values,or the difference values. The difference values will be used when discussing thecomputer output.

The estimated mean of the difference amounts in the universe is the sample mean, $70.60. x =

The estimated total difference for the universe (T) is the sample mean times the universe size;

that is, (70.60)(10,000) = $706,000. This is referred to as the POINT ESTIMATE in the$T =

computer output.

The sample standard deviation is s = 48.2519 and the corresponding (estimated) standard error

for the mean is The (estimated) standard error for the48 251910000 5050 10000

680677.( )( )

. .−

=

total is 10,000 x 6.80677 = 68,067.7 (68,068 rounded). The sample skewness is a measure of

the symmetry of the sample data. This value is SKEWNESS = 0.64, indicating a very slight

positive (right-tail) skew. The sample kurtosis is a measure of the sample “peakedness” and is

equal to KURTOSIS = 2.98. Essentially, this value is small whenever the frequency of

observations close to the mean is high and the frequency of observations far from the mean is

low.

The 95% confidence interval for the universe total of the difference amounts is 706,000 ±

(2.009575237129)(68,067.7) = 706,000 ± 136,787; that is, 569,213 to 842,787. The

PRECISION AMOUNT is the amount added and subtracted to the POINT ESTIMATE; that is,

$136,787. This value is 19.37% of the point estimate and is referred to as the PRECISION

PERCENT.1


(Rev. 10/2004) Page 3-7

FORMULAS

STANDARD DEVIATION = s =

( )x x

n

ii

n

−

−=∑ 2

11

STANDARD ERROR (MEAN) = and STANDARD ERROR (TOTAL) = sN nnN− Ns N n

nN−

where n = sample size, N = universe size.

SKEWNESS =

1

1

3

1

2

1

3 2

nx x

nx x

i

n

i

n

( )

( )

/

−

−⎡

⎣

⎢⎢

⎤

⎦

⎥⎥

=

=

∑

∑

KURTOSIS =

1

1

4

1

2

1

2

nx x

nx x

i

n

i

n

( )

( )

−

−⎡

⎣

⎢⎢

⎤

⎦

⎥⎥

=

=

∑

∑

95% confidence interval for the universe total (T)

$ ( ). ,T t s N N n

nn± ⋅ ⋅−

−025 1

where (1) $T x N= ⋅

(2) t.025,n-1 is the t-value with n - 1 df having a right-tail area = .025 (RAT-STATSprovides t-values accurate to 12 decimal places).

NOTE: For a 90% confidence interval, replace t.025,n-1 with t.05,n-1 and for an 80% confidenceinterval, replace t.025,n-1 with t.10,n-1.

Stratified Variable Appraisal RAT-STATS Companion Manual

Page 3-8 (Rev. 10/2004)

Stratified Variable Appraisal

In a stratified variable sampling plan, the universe is divided into two or more nonoverlapping

categories (strata). As with an unrestricted sample, the intent is to make a statistical estimate for

a universe total (T) for a particular variable of interest. This plan involves obtaining a random

sample from each of the strata. The program will request the number of universe items in each

stratum and these values must be known. The program will develop estimates for each stratum

as well as for the entire universe.

Using a Stratified Sample

Purpose: To divide (partition) the universe into separate strata so that variation withinindividual strata is less than variation within the entire universe.

Simple Illustration:

Universe consists of {5 7 8 10 55 60 66 70 120 133 145 150}

Mean of universe is : = 69.08

Variance of universe is F2 = 2871.9

Partition the universe into three strata:

{5 7 8 10} {55 60 66 70} {120 133 145 150}#1 #2 #3

The strata variances are: Stratum Variance 1 3.25 2 32.69 µ Compare these to F2 = 2871.9 3 134.50

Consequently, the individual strata are much more homogeneous.

So, when obtaining a stratified sample, the user can take a larger sample (perhaps 100%) from

the stratum containing the large dollar items.

RAT-STATS Companion Manual Stratified Variable Appraisal

(Rev. 10/2004) Page 3-9

Reasons for Using Stratified Sampling:

A. Improved Sampling Precision

Stratification tends to make the sampling more efficient; that is, the user will obtain narrower

confidence intervals for the same sample size. When a sample is skewed or has a high degree of

variability, the sample size required to provide a reasonable degree of precision using simple

random sampling may be quite large. Precision is improved because each stratum should have a

relatively small variance and the weighted sum of the strata variances is less than the variance

for the entire universe.

B. Separate Information About Strata and the Universe

Strata may be formed because separate estimates are desired for subuniverses. For example, a

nationwide audit of nursing homes can be planned in advance such that separate estimates are

published for each state (stratum). When an auditor selects a simple random sample from the

entire universe, he/she cannot control the sample size within each stratum. Stratified sampling

permits the auditor to also impose different precision requirements on different strata, such as

requiring more precise estimates for large accounts.

C. Accommodation of Different Techniques

It may be desirable to employ different sampling methods or audit techniques in various portions

of the universe. For example, in a sample of health service employees, the headquarters

employees (Stratum 1) may be sampled as individuals and the employees scattered throughout

the state (Stratum 2) may be sampled as clusters to save travel time and cost.


Page 3-10 (Rev. 10/2004)

Comments:

(1) Defining effective strata is no accident! The user can incorporate all sorts of prior

knowledge in defining the strata. Such a technique does not introduce any bias into the

final estimate since strata are defined prior to obtaining the sample and each sampling item

has a known (although not the same) chance of being selected. As a result, a well-designed

stratified plan can provide audit protection and/or improved precision.

(2) Strata can be defined after sample data are obtained provided the proportions of the

universe in each stratum are known (with negligible error) and samples of at least 20 are

obtained from each stratum.

(3) Generally, it is not a good idea to stratify for convenience (unlike cluster or multistage

sampling) since the resulting estimator may be less efficient than the estimator which uses

a single simple random sample.

(4) Even though random selection is performed within strata, this does not mean that the user

cannot take a close look at the individual findings to determine nature, source, cause, trend

and impact.

(5) A careful balance must be maintained between the gains expected in sample precision and

the additional time and resources involved in introducing a stratified scheme into the

sample design.


(Rev. 10/2004) Page 3-11

Strata Formation

Strata are typically defined using the dollar value of the items being sampled. An alternative is

to stratify using some other variable which is highly correlated with the principal variable, such

as using the number of hospital beds to measure the “size” of a hospital.

Basic Rule: Select strata so that their means are as different as possible and their standarddeviations are as small as possible.

Guidelines:

• A few strata yield most of the gains (say, 2 to 6).

• Experience, intuition, and the judgment of the auditor are extremely useful in improvingthe sampling precision through effective stratification.

• Quantitative rather than qualitative (sex, race, etc.) variables are preferable for definingstrata.

• Coarser divisions of several stratifying variables are preferable to finer divisions of onevariable.

• It is better to use unrelated stratifying variables.

Example 2. Random samples of size 25 were obtained from two strata:

Stratum 1: Examined amounts under $200 (N1 = 5,200) and

Stratum 2: Examined amounts $ $200 (N2 = 3,500)

NOTE: These sample sizes are too small to meet OAS standards and are used for illustrativepurposes only.

The sample difference amounts for the two strata are stored in data file DATASTRAT.TXT and

the universe/sample sizes are stored in file UNIVSTRAT.TXT.


Page 3-12 (Rev. 10/2004)

Data set DATASTRAT.TXT Universe File UNIVSTRAT.TXT1 80 1 5200 252 43 2 3500 253 1334 1255 1166 847 1118 1489 10410 11411 8312 13213 9614 8615 6616 8917 7218 11419 13520 7121 12722 10523 10224 6925 7626 35427 32828 31329 25030 26131 29432 38033 29634 24835 27736 33137 30538 36039 34840 31841 29042 24943 36244 34845 35546 29547 27748 35549 31450 277


(Rev. 10/2004) Page 3-13

The sample results are:

Stratum 1: n1 = 25, mean = 99.24, std. dev. = 26.3317

Stratum 2: n2 = 25, mean = 311.40, std. dev. = 39.6432

The following computer output was obtained from the Stratified Variables Appraisal program:

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 4/5/2004 STRATIFIED VARIABLE APPRAISAL Time: 12:17 AUDIT/REVIEW: Variable - Stratified

DATA FILE USED: C:\Temp\DATASTRAT.TXT

STRATUM SAMPLE NUMBER SIZE VALUE OF SAMPLE NONZERO ITEMS 1 25 2,481.00 25 2 25 7,785.00 25

TOTALS 50 10,266.00 50

--------------------- D I F F E R E N C E ----------------------Stratum 1 MEAN / UNIVERSE 99.24 5,200 STANDARD DEVIATION 26.33 SKEWNESS -.07 KURTOSIS 2.24 STANDARD ERROR (MEAN) 5.25 STANDARD ERROR (TOTAL) 27,319 POINT ESTIMATE 516,048





Page 3-14 (Rev. 10/2004)

Stratum 2 MEAN / UNIVERSE 311.40 3,500 STANDARD DEVIATION 39.64 SKEWNESS -.06 KURTOSIS 1.85 STANDARD ERROR (MEAN) 7.90 STANDARD ERROR (TOTAL) 27,651 POINT ESTIMATE 1,089,900




OVERALL POINT ESTIMATE / UNIVERSE 1,605,948 8,700 STANDARD ERROR 38,870

CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT 1,556,134 UPPER LIMIT 1,655,762 PRECISION AMOUNT 49,814 PRECISION PERCENT 3.10% Z-VALUE USED 1.281551565545

90% CONFIDENCE LEVEL LOWER LIMIT 1,542,012 UPPER LIMIT 1,669,884 PRECISION AMOUNT 63,936 PRECISION PERCENT 3.98% Z-VALUE USED 1.644853626951



(Rev. 10/2004) Page 3-15

Discussion. The point estimates for the universe total difference amounts are $516,048

(Stratum 1) and $1,089,900 (Stratum 2). Referring to the formula section and the OVERALL

section in the computer output, the estimate of the universe total difference is

= (5200)(99.24) + (3500)(311.40) = $1,605,948$T

The estimated variance of is$T

= 1,510,906,28752005200 25

520026 3317

253500

3500 253500

39 643225

22

22−⎛

⎝⎜⎞⎠⎟ +

−⎛⎝⎜

⎞⎠⎟

. .

The (estimated) standard error of is $T 1510 906 287 38 870, , , , .=

The 95% confidence interval for universe total (T) is

1,605,948 ± (1.959963984540)(38,870)

that is, 1,605,948 ± 76,184 ($1,529,764 to $1,682,132).

The PRECISION AMOUNT here is $76,184 and is 4.74% of the point estimate, .$T

NOTE: When the POINT ESTIMATE is negative, the PRECISION PERCENT is set equal tozero.

FORMULAS

NOTE: For definitions and formulas of the statistics within each stratum (standard deviation,standard error, skewness, and kurtosis), refer to the previous section (UnrestrictedVariable Appraisal).

1. Estimate of universe mean (:):

y N N y N N y N N yst L L= + + ⋅ ⋅ ⋅ +( / ) ( / ) ( / )1 1 2 2


Ni = number of items in i-th stratum (universe)


Page 3-16 (Rev. 10/2004)

N = N1 + N2 + @@@ + NL

= average of sample items in the i-th stratumyi

2. Estimate of universe total (T):$T N y N y N y N yst L L= ⋅ = ⋅ + ⋅ + ⋅ ⋅ ⋅ + ⋅1 1 2 2

3. Estimated variance of :yst

v yN

NN n

Nsnst i

i

Li i

i

i

i( ) =

−⎛⎝⎜

⎞⎠⎟

=∑1

22

1

2

where ni = number of sampled items in i-th stratum

si2 = sample variance for i-th stratum

4. Estimated variance of :$Tv T N v yst( $) ( )= 2

5. Approximate 95% confidence interval for universe mean (:):

where Z.025 = 1.959963984540.y Z v yst st± . ( )025

6. Approximate 95% confidence interval for universe total (T):

$ ( $).T Z v T± 025

NOTES: 1. For a 90% confidence interval, replace Z.025 with Z.05 = 1.644853626951 and foran 80% confidence interval, replace Z.025 with Z.10 = 1.281551565545.

2. The confidence intervals for each stratum total use t-values that are accurate to12 decimal places.

RAT-STATS Companion Manual Two-Stage Unrestricted Variable Appraisal

(Rev. 10/2004) Page 3-17

Two-Stage Unrestricted Variable Appraisal

This is a special case of multistage sampling. Multistage sampling is a very cost-effective

sampling procedure when (1) obtaining a frame that lists all elements in the universe is very

costly or impossible or (2) the cost of obtaining observations increases as the distance separating

the elements increases. Put another way, multistage sampling is cost effective when it is more

costly to get to the sampling unit than it is to audit the sampling unit. The goal of multistage

sampling is to get the most precise results per unit of examination cost.

General Comments

1. This is a very convenient sampling procedure for many situations. You don't have to visit all

locations.

2. For a two-stage procedure, the universe can be broken down into “subgroups.”

Example: 1st Stage: Carriers (primary units, P.U.s)

2nd Stage: Hospitals (secondary units, S.U.s)

So, the procedure is to first obtain a random sample of P.U.s. These are called clusters.

Then, obtain a random sample of S.U.s within each selected P.U. Notice that at the first

stage, clusters are the sampling unit (sampling units are not always individual people,

records, etc.). The program will accept a maximum of 20 clusters.

3. You can estimate cost recoveries for the entire universe with multistage sampling and it is

very useful for large, widespread universes.


research grants. Because these universities are so widespread, it was decided to use a cluster

Two-Stage Unrestricted Variable Appraisal RAT-STATS Companion Manual

Page 3-18 (Rev. 10/2004)

sample of n = 10 universities. The 10 universities to be sampled may be obtained using the

Single-Stage Random Numbers module discussed in the Random Numbers section.

Enter the values shown in the following input screen:

The resulting output is shown on the next page. Note that the selected universities are in

sequential order:

Universities: 2, 5, 7, 23, 28, 46, 56, 67, 70, 76


(Rev. 10/2004) Page 3-19

Department of Health and Human Services OIG - Office of Audit ServicesDate: 10/26/2004 Random Number Generator Time: 11:03 AUDIT: select

SEED NUMBER: 1357.00 FRAME SIZE: 90

FILE OF RANDOM NUMBERS: C:\TEMP\SELECT.TXT

TOTAL RANDOM NUMBERS GENERATED: 10

THE NUMBERS ARE IN THE FOLLOWING FORMAT IN YOUR FILE: POSITIONS 1 THROUGH 6 - ORDER OF SELECTION POSITIONS 7 THROUGH 17 - RANDOM NUMBEREACH COLUMN OF NUMBERS IS RIGHT JUSTIFIED.

Selection Order Value 4 2 7 5 9 7 10 23 5 28 2 46 6 56 1 67 8 70 3 76

SUMMATION OF RANDOM NUMBERS = 380

Example 3 -- continued. Rather than audit all grants at a selected university, it was decided

(based on available resources) to audit roughly 20% of the grants at each selected university.

There are a total of M = 4500 grants in all 90 universities (see NOTE 5 under Formula 2). The

following data were obtained where yi is the dollars (in thousands) of improper charges for the

ith

sampled university, Mi is the total number of grants at the i-th university, and mi is the number of

audited grants at the i-th university. Also, and are the mean and variance of the sampleyi si2

values from the i-th university.


Page 3-20 (Rev. 10/2004)

For ease of illustration, university 1 refers to university 2, university 2 refers to university 5, and

so on.

Univ. Mi mi Dollars (yi, in thousands) yi si2 1 50 10 5, 7, 9, 0, 11, 2, 8, 4, 3, 5 5.40 11.38 2 65 13 4, 3, 7, 2, 11, 0, 1, 9, 4, 3, 2, 1, 5 4.00 10.67 3 45 9 5, 6, 4, 11, 12, 0, 1, 8, 4 5.67 16.75 4 48 10 6, 4, 0, 1, 0, 9, 8, 4, 6, 10 4.80 13.29 5 52 10 11, 4, 3, 1, 0, 2, 8, 6, 5, 3 4.30 11.12 6 58 12 12, 11, 3, 4, 2, 0, 0, 1, 4, 3, 2, 4 3.83 14.88 7 42 8 3, 7, 6, 7, 8, 4, 3, 2 5.00 5.14 8 66 13 3, 6, 4, 3, 2, 2, 8, 4, 0, 4, 5, 6, 3 3.85 4.31 9 40 8 6, 4, 7, 3, 9, 1, 4, 5 4.88 6.13 10 56 11 6, 7, 5, 10, 11, 2, 1, 4, 0, 5, 4 5.00 11.80

NOTE: This example violates OAS minimum sample sizes of at least 30 grants at eachuniversity. It is used for illustration only. These data are in data setDATA2STG.TXT. The universe sizes (the Mi's) are in data set UNIV2STG.TXT. The program output immediately follows.

Dataset DATA2STG.TXT Dataset UNIV2STG.TXT1 5 1 50 102 7 2 65 133 9 3 45 94 0 4 48 105 11 5 52 106 2 6 58 127 8 7 42 88 4 8 66 139 3 9 40 810 5 10 56 1111 412 313 714 215 1116 017 118 919 420 321 222 1


(Rev. 10/2004) Page 3-21

23 5 . . .94 695 796 597 1098 1199 2100 1101 4102 0103 5104 4

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/29/2004 TWO-STAGE UNRESTRICTED VARIABLE APPRAISAL Time: 10:07 AUDIT/REVIEW: Variable 2-Stage

DATA FILE USED: C:\Temp\DATA2STG.TXT

--------------------- D I F F E R E N C E ----------------------UNIT SAMPLE SIZE/ NBR NONZERO ITEMS SAMPLE MEAN VARIANCE UNIVERSE SIZE POINT ESTIMATE

1 10/9 5.40 11.38 50 270 2 13/12 4.00 10.67 65 260 3 9/8 5.67 16.75 45 255 4 10/8 4.80 13.29 48 230 5 10/9 4.30 11.12 52 224 6 12/10 3.83 14.88 58 222 7 8/8 5.00 5.14 42 210 8 13/12 3.85 4.31 66 254 9 8/8 4.88 6.13 40 195 10 11/10 5.00 11.80 56 280

104/94 4.80 522 2,400

NOT SAMPLED 80 3,978OVERALL TOTALS 90 4,500 21,602

STANDARD ERROR 867 CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT 20,491 UPPER LIMIT 22,712 PRECISION AMOUNT 1,111 PRECISION PERCENT 5.14% Z-VALUE USED 1.281551565545


Page 3-22 (Rev. 10/2004)

90% CONFIDENCE LEVEL LOWER LIMIT 20,176 UPPER LIMIT 23,027 PRECISION AMOUNT 1,425 PRECISION PERCENT 6.60% Z-VALUE USED 1.644853626951


Discussion. The point estimate (highlighted) for the universe total difference amount is 21,602

(that is, $21,602,000). The 95% confidence interval for this amount is from 19,903 to 23,300

($19,903,000 to $23,300,000). The PRECISION AMOUNT at the 95% confidence level is

(Z-value)(standard error of ) = (1.959963984540)(867) = 1,699 (that is, $1,699,000). This value$T

is 7.86% of the point estimate.

Notice that the output also contains the estimated totals for each primary unit (university). For

example, the estimated total difference for university 1 is 270 ($270,000). The sample average of

these estimates is 2,400/10 = 240 ($240,000). Since there are 90 universities in the universe, the

point estimate for the universe total is (90)(240) = 21,600; that is, $21,600,000 (more precisely,

21,602,000).

FORMULAS

1. The point estimate for the universe total (T) is

$T N

M y

n

i ii

n

= =∑

1


(Rev. 10/2004) Page 3-23

2. The estimated variance of is $T

v TN N n

n

M yTN

nNn

MM m

Msm

i ii

n

ii i

i

i

ii

n

( $)( )

$

=−⎛

⎝⎜⎞⎠⎟

−⎛

⎝⎜

⎞

⎠⎟

−

⎛

⎝

⎜⎜⎜⎜⎜⎜

⎞

⎠

⎟⎟⎟⎟⎟⎟

+−⎛

⎝⎜

⎞⎠⎟=

=

∑∑1

2

22

11

NOTES: 1. n = number of primary units in the sample and N = number of primary units in the

universe.

2. The STANDARD ERROR of is the square root of v( ).$T $T

3. The PRECISION AMOUNT at the 95% confidence level for the universe total is

(1.959963984540)(standard error of ).$T

4. For the PRECISION AMOUNT at the 90% confidence level, replace

1.959963984540 with 1.644853626951. For the PRECISION AMOUNT at the

80% confidence level, replace 1.959963984540 with 1.281551565545.

5. The total number of secondary units in the universe (M) may be known or

unknown and is not used in any of the calculations.

3. The approximate 95% confidence interval for T is

$ . ( $)T v T± 1959963984540

NOTE: For a 90% confidence interval, replace 1.959963984540 with 1.644853626951 andfor an 80% confidence interval replace 1.959963984540 with 1.281551565545.

Three-Stage Unrestricted Variable Appraisal RAT-STATS Companion Manual

Page 3-24 (Rev. 10/2004)

Three-Stage Unrestricted Variable Appraisal

Example 4. The situation discussed in Example 3 was extended the following year to a three-

stage procedure by defining:




NOTE: This example violates OAS minimum sample sizes and is used for illustration only.

Using the Single-Stage Random Numbers module, regions 5, 7, 8, and 10 were selected as the

sampled primary units. Next, 10 universities (secondary units) were randomly selected from the

available universities in each of the four selected regions. The data on the following four pages

were obtained, where Mi is the number of grants in the universe for each university, mi is the

number of sampled grants at each university (chosen to be roughly 20% of Mi),

is the sample average of the items from the j-th university within the i-th region, and isyij sij

the sample standard deviation of the items from the j-th university within the i-th region. The

resulting data are stored in file DATA3STG.TXT.

REGION 5 -- Universe contains 90 universities.

Univ. Mi mi 1 47 9 2 51 10 3 45 9 4 46 9 5 46 9 6 50 10 7 50 10 8 57 11 9 54 11 10 64 13

Data (total thousands of dollars of improper charges)

RAT-STATS Companion Manual Three-Stage Unrestricted Variable Appraisal

(Rev. 10/2004) Page 3-25

University Obs. 1 2 3 4 5 6 7 8 9 10

1 * 8 13 10 11 14 5 0 0 2 12 2 * 0 13 0 9 14 11 14 3 9 13 3 * 6 4 12 12 5 0 4 0 15 14 4 * 6 6 14 1 1 4 10 6 4 11 5 * 0 0 13 10 2 13 13 1 13 6 6 * 13 15 13 11 11 8 10 0 12 10 7 * 1 12 13 15 14 15 8 3 14 11 8 * 7 9 2 7 15 2 0 13 6 0 9 * 2 0 9 8 11 0 7 7 11 7 10 * 13 14 3 5 0 12 11 * 4 1 9 12 * 11 13 * 7

4.78 8.50 9.56 9.33 9.67 7.20 6.90 3.82 7.91 9.46y j1 4.38 5.64 5.13 3.91 5.52 5.79 5.02 3.92 5.52 3.73s j1


Univ. Mi mi 1 53 11 2 59 12 3 52 10 4 67 13 5 59 12 6 73 15 7 51 10 8 75 15 9 66 13 10 58 12


University Obs. 1 2 3 4 5 6 7 8 9 10

1 * 0 12 4 5 0 2 4 19 17 16 2 * 4 0 19 10 4 18 16 13 8 17 3 * 0 15 16 2 6 0 0 0 15 6 4 * 10 11 12 10 9 0 8 13 12 10 5 * 11 0 4 12 13 12 3 4 0 2 6 * 18 18 2 7 19 17 8 0 20 6 7 * 18 0 1 3 9 0 13 0 17 13 8 * 16 17 5 0 18 0 0 0 6 0 9 * 2 8 1 20 16 16 0 0 9 12


Page 3-26 (Rev. 10/2004)

10 * 18 12 8 10 15 3 9 13 16 20 11 * 18 13 0 0 0 4 7 19 12 * 5 0 13 4 16 2 0 13 * 15 3 13 7 14 * 6 0 15 * 5 4

10.45 9.25 7.20 7.23 10.17 5.73 6.10 6.60 10.46 10.08y j2

7.69 6.59 6.41 6.30 6.59 6.65 5.65 7.01 6.21 7.23s j2


Univ. Mi mi 1 45 9 2 39 8 3 43 9 4 34 7 5 54 11 6 54 11 7 34 7 8 59 12 9 49 10 10 43 9


University Obs. 1 2 3 4 5 6 7 8 9 10 1 * 6 0 10 4 13 0 13 0 6 14 2 * 5 8 15 1 13 0 10 13 8 0 3 * 1 0 11 2 15 14 10 9 3 0 4 * 3 1 6 5 10 11 6 3 12 10 5 * 12 10 12 2 10 14 9 14 4 12 6 * 7 15 2 9 7 5 8 14 8 0 7 * 0 1 14 4 0 1 1 12 1 7 8 * 3 14 7 0 13 11 2 6 9 * 12 0 3 0 11 4 0 10 * 13 5 6 3 11 * 1 10 15 12 * 11

5.44 6.12 8.56 3.86 7.73 6.64 8.14 9.92 5.10 5.44y j3

4.33 6.40 5.20 2.67 5.78 5.90 3.80 4.66 3.38 5.68s j3


(Rev. 10/2004) Page 3-27


Univ. Mi mi 1 59 12 2 68 14 3 57 11 4 72 14 5 70 14 6 73 15 7 83 17 8 89 18 9 73 15 10 77 15


University Obs. 1 2 3 4 5 6 7 8 9 10

1 * 0 3 8 9 7 15 18 16 20 18 2 * 10 13 10 10 6 15 10 2 14 0 3 * 14 0 1 15 3 7 17 14 5 8 4 * 0 12 8 16 15 12 18 0 0 4 5 * 18 12 19 6 12 15 0 0 19 5 6 * 0 7 18 17 10 4 0 0 0 17 7 * 8 1 15 5 17 0 8 6 12 0 8 * 20 13 0 2 0 14 0 19 5 0 9 * 19 2 0 8 0 18 3 17 15 13 10 * 0 0 17 9 0 17 18 13 15 8 11 * 0 16 18 16 0 13 12 12 2 10 12 * 3 14 7 0 6 0 13 15 7 13 * 17 0 7 0 11 12 2 10 14 * 5 0 16 3 19 11 2 0 15 * 4 0 14 1 0 16 * 9 13 17 * 6 0 18 * 1

7.67 8.21 10.36 8.57 6.64 9.53 8.76 9.06 8.47 6.67y j4

8.25 6.27 7.55 5.80 6.44 6.32 7.38 6.77 7.39 6.17s j4

Using the file construction suggested in the User’s Guide for this module, the primary unit file

and secondary unit file could be constructed as shown below. In the secondary unit file, each

line begins with a counter (1, 2, 3, ...), a value identifying the primary unit number in the second


Page 3-28 (Rev. 10/2004)

column, and a value identifying the secondary unit number within each primary unit in the third

column.

Data set PRIMARY3STG.TXT1 REGION 5 90 102 REGION 7 110 103 REGION 8 85 104 REGION 10 120 10

Data set SECONDARY3STG.TXT1 1 1 UNIV1 47 92 1 2 UNIV2 51 103 1 3 UNIV3 45 94 1 4 UNIV4 46 95 1 5 UNIV5 46 96 1 6 UNIV6 50 107 1 7 UNIV7 50 108 1 8 UNIV8 57 119 1 9 UNIV9 54 1110 1 10 UNIV10 64 1311 2 1 UNIV1 53 1112 2 2 UNIV2 59 1213 2 3 UNIV3 52 1014 2 4 UNIV4 67 1315 2 5 UNIV5 59 1216 2 6 UNIV6 73 1517 2 7 UNIV7 51 1018 2 8 UNIV8 75 1519 2 9 UNIV9 66 1320 2 10 UNIV10 58 1221 3 1 UNIV1 45 922 3 2 UNIV2 39 823 3 3 UNIV3 43 924 3 4 UNIV4 34 725 3 5 UNIV5 54 1126 3 6 UNIV6 54 1127 3 7 UNIV7 34 728 3 8 UNIV8 59 1229 3 9 UNIV9 49 1030 3 10 UNIV10 43 931 4 1 UNIV1 59 1232 4 2 UNIV2 68 1433 4 3 UNIV3 57 1134 4 4 UNIV4 72 1435 4 5 UNIV5 70 1436 4 6 UNIV6 73 1537 4 7 UNIV7 83 1738 4 8 UNIV8 89 1839 4 9 UNIV9 73 1540 4 10 UNIV10 77 15


(Rev. 10/2004) Page 3-29

Once again using the file construction suggested in the User’s Guide for this module, the data

file could be constructed as shown below. The data lines for the first two universities in Region

5 and the last two universities in Region 10 are shown. Each line begins with a counter (1, 2, 3,

...), a value identifying the primary unit number in the second column, a value identifying the

secondary unit number within each primary unit in the third column, and a value identifying the

third-stage unit number within each sampled primary/secondary unit in the fourth column. The

sample value appears in the fifth column.

Data set DATA3STG.TXT1 1 1 1 82 1 1 2 03 1 1 3 64 1 1 4 65 1 1 5 06 1 1 6 137 1 1 7 18 1 1 8 79 1 1 9 210 1 2 1 1311 1 2 2 1312 1 2 3 413 1 2 4 614 1 2 5 015 1 2 6 1516 1 2 7 1217 1 2 8 918 1 2 9 019 1 2 10 13 . . .433 4 9 1 20434 4 9 2 14435 4 9 3 5436 4 9 4 0437 4 9 5 19438 4 9 6 0439 4 9 7 12440 4 9 8 5441 4 9 9 15


Page 3-30 (Rev. 10/2004)

442 4 9 10 15443 4 9 11 2444 4 9 12 15445 4 9 13 2446 4 9 14 2447 4 9 15 1448 4 10 1 18449 4 10 2 0450 4 10 3 8451 4 10 4 4452 4 10 5 5453 4 10 6 17454 4 10 7 0455 4 10 8 0456 4 10 9 13457 4 10 10 8458 4 10 11 10459 4 10 12 7460 4 10 13 10461 4 10 14 0462 4 10 15 0

The program output using these three files is shown on the following pages.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/4/2004 THREE-STAGE UNRESTRICTED VARIABLE APPRAISAL Time: 17:34 AUDIT/REVIEW: Variable 3-Stage

DATA FILE USED: C:\Temp\DATA3STG.TXT

--------------------- D I F F E R E N C E ----------------------FIRST STAGE SAMPLE NON- SECOND STAGE SIZE ZEROES SAMPLE MEAN VARIANCE UNIVERSE POINT ESTIMATE================ ====== ====== =========== ======== ========== ==============REGION 5 UNIV1 9 7 4.78 19.19 47 225 UNIV2 10 8 8.50 31.83 51 434 UNIV3 9 8 9.56 26.28 45 430 UNIV4 9 9 9.33 15.25 46 429 UNIV5 9 9 9.67 30.50 46 445 UNIV6 10 8 7.20 33.51 50 360 UNIV7 10 8 6.90 25.21 50 345 UNIV8 11 8 3.82 15.36 57 218 UNIV9 11 10 7.91 30.49 54 427 UNIV10 13 12 9.46 13.94 64 606

COMBINED 101 392 90 35,256


(Rev. 10/2004) Page 3-31

REGION 7 UNIV1 11 9 10.45 59.07 53 554 UNIV2 12 9 9.25 43.48 59 546 UNIV3 10 10 7.20 41.07 52 374 UNIV4 13 10 7.23 39.69 67 484 UNIV5 12 10 10.17 43.42 59 600 UNIV6 15 10 5.73 44.21 73 419 UNIV7 10 7 6.10 31.88 51 311 UNIV8 15 9 6.60 49.11 75 495 UNIV9 13 12 10.46 38.60 66 690 UNIV10 12 10 10.08 52.27 58 585

COMBINED 123 506 110 55,643

REGION 8 UNIV1 9 8 5.44 18.78 45 245 UNIV2 8 6 6.12 40.98 39 239 UNIV3 9 8 8.56 27.03 43 368 UNIV4 7 7 3.86 7.14 34 131 UNIV5 11 9 7.73 33.42 54 417 UNIV6 11 8 6.64 34.85 54 358 UNIV7 7 7 8.14 14.48 34 277 UNIV8 12 11 9.92 21.72 59 585 UNIV9 10 10 5.10 11.43 49 250 UNIV10 9 5 5.44 32.28 43 234

COMBINED 93 310 85 26,388

--------------------- D I F F E R E N C E ----------------------FIRST STAGE SAMPLE NON- SECOND STAGE SIZE ZEROES SAMPLE MEAN VARIANCE UNIVERSE POINT ESTIMATE================ ====== ====== =========== ======== ========== ==============REGION 10 UNIV1 12 7 7.67 68.06 59 452 UNIV2 14 12 8.21 39.26 68 559 UNIV3 11 9 10.36 57.05 57 591 UNIV4 14 12 8.57 33.65 72 617 UNIV5 14 9 6.64 41.48 70 465 UNIV6 15 13 9.53 39.98 73 696 UNIV7 17 12 8.76 54.44 83 727 UNIV8 18 14 9.06 45.82 89 806 UNIV9 15 13 8.47 54.55 73 618 UNIV10 15 10 6.67 38.10 77 513

COMBINED 145 604 120 72,534

STAGES UNIVERSE SAMPLED FIRST 12 4 SECOND 405 40 THIRD 2,298 462

OVERALL POINT ESTIMATE 569,464OVERALL STANDARD ERROR 102,337

CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT 438,314 UPPER LIMIT 700,615 PRECISION AMOUNT 131,151 PRECISION PERCENT 23.03% Z-VALUE USED 1.281551565545


Page 3-32 (Rev. 10/2004)



Some highlighted values: (1) 405 is 90 + 110 + 85 + 120, (2) 2,298 is the total number of third-

stage units (universe) for the four sampled primary units, (3) 462 is 101 + 123 + 93 + 145.

The point estimate and confidence intervals: The point estimate (highlighted) for the universe

total difference amount is 569,464 (that is, $569,464,000). The 95% confidence interval for this

amount is from 368,887 to 770,042 ($368,887,000 to $770,042,000). The PRECISION

AMOUNT at the 95% confidence level is (Z-value)(standard error of ) =$T

(1.959963984540)(102,337) = 200,578 (that is, $200,578,000). This value is 35.22% of the

point estimate.

Notice that the output also contains the estimated totals for each primary unit (region) and each

secondary unit (university). For example, the estimated total difference for UNIV1 within

region 5 is 225 ($225,000). The sample average of the 10 university estimates in region 5 is 392

($392,000). Since there are 90 universities in this region, the point estimate for the region 5 total

is (90)(392) = 35,280 (more precisely, 35,256 or $35,256,000).

The average of the four regional estimates is (35,256 + 55,643 + 26,388 + 72,534)/4 = 47,455.25

($47,455,250). Since there are 12 regions in the universe, the (unbiased) point estimate for the


(Rev. 10/2004) Page 3-33

universe total is (12)(47,455,250) = approximately $569,463,000. The actual amount

(highlighted) is $569,464,000.

FORMULAS

1. Point estimate of the universe total (T):

$ $TNn

Tii

n

==∑

1

where is the estimate of the total for the i-th sampled P.U. and where$ $TMm

Tii

iij

j

mi

==∑

1

is the estimate of the total for the j-th S.U. within the i-th sampled P.U.$TBb

yijij

ijijk

k

bij

==∑

1

Notation: n = number of primary units (P.U.s) in the sample

N = number of P.U.s in the universe

mi = number of sampled secondary units (S.U.s) in the i-th P.U. (i = 1, . . ., n)

Mi = number of S.U.s in the universe in the i-th P.U. (i = 1, . . ., n)

bij = number of 3rd-stage items in the sample for the i-th P.U. and j-th S.U.(i = 1, . . ., n and j = 1, . . ., mi)

Bij = number of 3rd-stage items in the universe for the i-th P.U. and j-th S.U.(i = 1, . . ., n and j = 1, . . ., mi)

yijk = sample value of the k-th item from the i-th P.U. and j-th S.U.(i = 1, . . ., n and j = 1, . . ., mi, k = 1, . . ., bij)

NOTE: The value of n, N, along with mi, Mi, bij, Bij for each sampled primary and secondaryunit must be known.

2. Estimated variance of :$T


Page 3-34 (Rev. 10/2004)

v TN N n

ns

Nn

M M mm

sNn

Mm

B B bb

si i i

ii

n

ii

ii

nij ij ij

ijj

m

ij

i

( $)( ) ( ) ( )

=−

+−

+−

= = =∑ ∑ ∑2

1

2

1 1

2

where s T T n ni ii

n

i

n2 2

1

2

11= −

⎛

⎝

⎜⎜

⎞

⎠

⎟⎟

⎡

⎣

⎢⎢⎢

⎤

⎦

⎥⎥⎥

−==∑∑ $ $ / / ( )

s T T m mi ij ijj

m

ij

m

i

ii

2 2

1

2

11= −

⎛

⎝

⎜⎜

⎞

⎠

⎟⎟

⎡

⎣

⎢⎢⎢

⎤

⎦

⎥⎥⎥

−==∑∑ $ $ / / ( )

s y y b bij ijk ijkk

b

ijk

b

ij

ijij

2 2

1

2

11= −

⎛

⎝

⎜⎜

⎞

⎠

⎟⎟

⎡

⎣

⎢⎢⎢

⎤

⎦

⎥⎥⎥

−==∑∑ / / ( )

3. The approximate 95% confidence interval for T:

$ . ( $)T v T± 1959963984540

NOTE: For a 90% confidence interval, replace 1.959963984540 with 1.644853626951 andfor an 80% confidence interval, replace 1.959963984540 with 1.281551565545.

RAT-STATS Companion Manual RHC Two-Stage Variable Appraisal

(Rev. 10/2004) Page 3-35

RHC Two-Stage Variable Sampling

For a discussion on the motivation behind the RHC (developed by Rao, Hartley, and Cochran)

sampling procedure, refer to the RHC Sample Selection section, contained in the Random

Numbers section of this manual. It provides a method of sample selection that allows sampling

without replacement while maintaining the flavor of sampling using probability proportional to

size. When the primary units (P.U.s) are selected, the size of each P.U. is considered rather than

obtaining a simple random sample of P.U.s.

The size of each P.U. is rather arbitrary and can be the number of people, dollars, beds (for

hospitals), and so forth. In general, you can expect improved precision using the RHC procedure

if there is a high correlation between the size of each P.U. and the number of secondary units

(S.U.s) within each P.U.. In other words, larger P.U.s should contain a larger number of S.U.s.

The P.U.s are selected using the RHC Sample Selection program. A random sample is then

obtained for each selected P.U. and the variable(s) of interest (e.g., dollars in error) is/are

recorded.

Example 5. (Note: This is the same example used in Example 8 in the RHC Sample Selection

discussion contained in the Random Numbers section). In a particular region of the United

Sates there are N = 90 universities (primary units) with Government research grants. Because

these universities are so widespread, it was decided to use a cluster sample of n = 10 universities.

Rather than audit all grants at a selected university, it was decided (based on available resources)

to audit roughly 20% of the grants at each selected university. As a measure of the size for each

university, the total grant dollars were used.

RHC Two-Stage Variable Appraisal RAT-STATS Companion Manual

Page 3-36 (Rev. 10/2004)

DATA: University ID, number of grants, total grant dollars (90 rows)

The data are contained in UNIVRHC.TXT.

OUTPUT: The 10 universities to use in the sample (see last page of computer output):

UNIV78, UNIV42, UNIV49, UNIV5, UNIV19,

UNIV38, UNIV62, UNIV28, UNIV60, UNIV75

Here there are 10 groups with 9 universities per group. The output file created by this program

is OUTRHC.TXT. This program is required by the RHC appraisal program.

Data set UNIVRHC.TXT and the program output are contained in the pages to follow.


(Rev. 10/2004) Page 3-37

Dataset UNIVRHC.TXT < - - - continued - - - > (1) (2)

(3)

< - - - continued - - - >




Columns: (1) unit ID(2) number of grants(3) grant dollar amount (x $100,000) µ This is the size of the university.


Page 3-38 (Rev. 10/2004)

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/15/2004 GENERATION OF PRIMARY UNIT SAMPLE Time: 12:52 NAME OF INPUT FILE: C:\TEMP\UNIVRHC.TXT




********* GROUP 2 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV6 9 43 UNIV42 <-- selected 12 63 UNIV65 7 32 UNIV40 18 77 UNIV45 10 51 UNIV1 8 42 UNIV80 10 49 UNIV36 5 24 UNIV70 7 31


< Groups 3 Through 9 Are Omitted Here >

********* GROUP 10 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV22 13 64 UNIV39 9 41 UNIV88 12 59 UNIV55 15 71 UNIV29 10 49 UNIV75 <-- selected 16 75 UNIV87 15 71 UNIV13 4 19 UNIV53 16 72



(Rev. 10/2004) Page 3-39




UNITS SECONDARY PRIMARY INPRIMARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP========================= ============= ============= ============= =====UNIV78 37 7 90 9UNIV42 63 12 86 9UNIV49 34 7 96 9UNIV5 51 11 84 9UNIV19 72 16 89 9UNIV38 76 17 89 9UNIV62 77 18 92 9UNIV28 78 18 115 9UNIV60 64 13 99 9UNIV75 75 16 110 9


Page 3-40 (Rev. 10/2004)

Example--continued. The sample size for each selected university (P.U.) was chosen to be

approximately 20% of the universe size. This leads to the following table where Mi is the total

number of grants at the i-th university, and mi is the number of audited grants at the i-th

university.

University Mi mi UNIV78 37 7 UNIV42 63 13 UNIV49 34 7 UNIV5 51 10 UNIV19 72 14 UNIV38 76 15 UNIV62 77 15 UNIV28 78 16 UNIV60 64 13 UNIV75 75 15 125

Data from these 125 secondary units (grants) were obtained by recording the total amount that

was charged to each grant after the scheduled completion of this grant (dollars in error). The

error amounts (in thousands of dollars) for each grant are contained in data set

DATARHC2.TXT.

The output file created using the sample selection program is also used as input to the appraisal

program in this two-step procedure. This file (PRIMRHC2.TXT), along with

DATARHC2.TXT, are used as the input files for the RHC Two-Stage appraisal program. Both

files are listed on the next page and the computer output from the appraisal program immediately

follows. The illustrated data file (DATARHC2.TXT) contains the data for the first two

universities (primary units) and the last two universities.


(Rev. 10/2004) Page 3-41

Dataset DATARHC2.TXT1 92 23 94 65 06 57 78 29 710 611 012 613 014 315 416 117 1318 819 020 621 1122 823 824 0 . . .100 7101 10102 2103 6104 0105 8106 4107 0108 10109 3110 2111 5112 10113 0114 0115 0116 0


Page 3-42 (Rev. 10/2004)

117 0118 8119 9120 0121 2122 8123 4124 6125 2

Output/Input file PRIMRHC2.TXTUNIV78 37 7 7 90 9UNIV42 63 13 12 86 9UNIV49 34 7 7 96 9UNIV5 51 10 11 84 9UNIV19 72 14 16 89 9UNIV38 76 15 17 89 9UNIV62 77 15 18 92 9UNIV28 78 16 18 115 9UNIV60 64 13 13 99 9UNIV75 75 15 15 110 9

NOTE: File PRIMRHC2.TXT was created by adding the third column containing samplesizes to the output file created by the RHC Sample Selection program.


(Rev. 10/2004) Page 3-43

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/26/2004 RHC TWO-STAGE VARIABLE APPRAISAL Time: 13:26 AUDIT/REVIEW: RHC 2-Stage

DATA FILE USED: C:\TEMP\DATARHC2.TXT

PRIMARY SAMPLE ==DIFFERENCE=== NUMBER OF UNIT SIZE SAMPLE TOTAL NONZERO ITEMS ======= ====== ============= ============= 1 7 38.00 6 2 13 56.00 10 3 7 33.00 4 4 10 55.00 8 5 14 67.00 13 6 15 76.00 14 7 15 79.00 14 8 16 84.00 13 9 13 61.00 11 10 15 54.00 9

TOTALS 125 603.00 102

PRIMARY UNIT FILE USED: C:\TEMP\PRIMRHC2.txt

UNITSP.U. SECONDARY PRIMARY INNBR PRIMARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP==== ========================= ============= ============= ============= ===== 1 UNIV78 37 7 90 9 2 UNIV42 63 12 86 9 3 UNIV49 34 7 96 9 4 UNIV5 51 11 84 9 5 UNIV19 72 16 89 9 6 UNIV38 76 17 89 9 7 UNIV62 77 18 92 9 8 UNIV28 78 18 115 9 9 UNIV60 64 13 99 9 10 UNIV75 75 15 110 9

TOTALS 627 134 950 90

* Note: 2,582 is the product of --- POINT ESTIMATES — 5.43, 37, and 12.857. This is the point estimate for P.U. 1. ==DIFFERENCE=== P.U. SAMPLE SECONDARY SIZES POINT NBR SIZE SAMPLE MEAN UNIVERSE RATIO ESTIMATE ==== ====== ============== ============= ======== ============= 1 7 5.43 37 12.857 2,582 * 2 13 4.31 63 7.167 1,945 3 7 4.71 34 13.714 2,198 4 10 5.50 51 7.636 2,142 5 14 4.79 72 5.563 1,917 6 15 5.07 76 5.235 2,016 7 15 5.27 77 5.111 2,073


Page 3-44 (Rev. 10/2004)

8 16 5.25 78 6.389 2,616 9 13 4.69 64 7.615 2,287 10 15 3.60 75 7.333 1,980

TOTALS: 125 627 21,756

Note: 21,756 is the point estimate

of the universe total ( ).$T --- VARIANCE COMPONENTS ---

P.U. WITHIN BETWEEN TOTAL NBR VARIANCE VARIANCE VARIANCE 1 23,689 283,356 307,044 2 25,870 659 26,529 3 38,797 0 38,797 4 22,443 53,230 75,674 5 19,701 15,570 35,271 6 13,977 523 14,500 7 33,107 1,192 34,299 8 29,738 247 29,985 9 23,704 369 24,074 10 31,366 247,927 279,293

TOTALS: 262,392 603,072 865,464

Note: 865,464 is equal to .v T( $)


POINT ESTIMATE OF POPULATION TOTAL: 21,756

STANDARD ERROR 930





(Rev. 10/2004) Page 3-45

Discussion: The (highlighted) estimate of the universe total on the previous page obtained

using formula 1 is:

= (Estimate of group 1 total) + AAA + (Estimate of group 10 total)$T

= (90/7)(5.43)(37) + (86/12)(4.31)(63) + AAA + (110/15)(3.60)(75)

= 2,582 + 1,945 + AAA + 1,980

= 21,756 ($21,756,000)

Using formula 2, the estimated variance of is:$T

= 865,464v T( $)

and the estimated standard error of is the square root of 865,464; that is, 930 (highlighted on$T

the previous page).

The approximate 95% confidence interval is:

21,756 ± 1.959963984540(930)

21,756 ± 1,823

that is, 19,933 to 23,579 ($19,933,000 to $23,579,000).


Page 3-46 (Rev. 10/2004)

FORMULAS

Definitions

1. P.U. stands for “primary unit” and S.U. is “secondary unit”

2. Ai = size of i-th P.U.

3. Si = (size of i-th P.U.)/(size of entire population) = Ai/(size of entire population)

4. Bi = total size for i-th group

5. Bi = (total size for i-th group)/(size of entire population) = Bi/(size of entire population)

6. N = number of P.U.s in the population

7. Ni = number of P.U.s in the i-th group

8. n = number of P.U.s in the sample

9. Mi = number of S.U.s in the i-th sampled P.U. (population)

10. mi = number of S.U.s in the i-th sampled P.U. (sample)

Estimate of population total (T)

$TBA

M yi

ii

n

i i=⎛⎝⎜

⎞⎠⎟

=∑

1

where = average of mi sampled S.U.s and Bi /Ai is labeled SIZES RATIO in the computeryi

output.

Estimated variance of T̂



(Rev. 10/2004) Page 3-47

V

N N

N N

M yS

Ti

i

n

ii

n ii i

ii

n

1

2

1

2 2

1

2

1=

−

−

−⎛⎝⎜

⎞⎠⎟=

=

=

∑

∑∑ π $

and

VMS

M mm

sii

i

i i

ii

n

i21

2=−⎛

⎝⎜

⎞⎠⎟

=∑π

where si2 = variance of the mi sampled S.U.s.

NOTE: The estimated standard error of is $T v T( $)


$ . ( $)T v T± 1959963984540


RHC Three-Stage Variable Appraisal RAT-STATS Companion Manual

Page 3-48 (Rev. 10/2004)

RHC Three-Stage Variable Sampling

The RHC sampling procedure can used for a three-stage design.

The steps for such a procedure are the following:

1. A sample of primary units (clusters) is obtained as in the one- and two-stage procedures. The size of the primary units is considered for this sample, where pps sampling is usedfor each group of primary units.

2. A sample of secondary units is obtained within each chosen primary unit by partitioningthe primary unit into random groups. The group sizes are chosen to be as nearly equal aspossible. Using pps sampling and the size of each secondary unit, one secondary unit ischosen from each of the secondary groups.

3. A random sample of third-stage units is obtained for each of the chosen secondary units. No attention is paid to “size” here. This is a random sample.

Prior to running the appraisal program, the user must run the RHC Sample Selection program in

the OAS software.

Example 6. The situation discussed in Example 4 in the Three-Stage Unrestricted section will

be appraised using the RHC methodology. For this example, the stages are:






(universities) in this region and (2) the size of this region (total dollars of grants). This file is

GRANTSPU.TXT. The selected regions are 4, 6, 8, and 10.

RAT-STATS Companion Manual RHC Three-Stage Variable Appraisal

(Rev. 10/2004) Page 3-49

NOTE: Seed values of 100 and 200 were used to select the primary units. In practice, it isrecommended that the user not set these seed values.

--- Data set GRANTSPU.TXT ---

(A) (B)REGION1 117 1250REGION2 63 610REGION3 91 720REGION4 123 1320REGION5 107 1160REGION6 116 1240REGION7 102 960REGION8 118 1300REGION9 122 1320REGION10 85 640REGION11 94 930REGION12 62 550

NOTE: It is okay to set the number of S.U.s [column (A)]equal to one in this file. The actual number ofS.U.s must be known for the selected P.U.s. Thecorrect number of S.U.s must then be inserted intofile GRANTSPUOUT.TXT (the highlightedvalues).

Columns: (A) number of universities (S.U.s)(B) size of each P.U. (total grant amount x $100,000)

--- Data set GRANTSPUOUT.TXT ---

REGION6 116 1240 3100 3REGION4 123 1320 3410 3REGION8 118 1300 3170 3REGION10 85 640 2320 3


Page 3-50 (Rev. 10/2004)

In practice, do not set these seed values.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 GENERATION OF PRIMARY UNIT SAMPLE Time: 15:09 NAME OF INPUT FILE: C:\TEMP\GRANTSPU.TXT




********* GROUP 2 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= REGION4 <-- Selected 1,320 123 REGION5 1,160 107 REGION11 930 94




********* GROUP 4 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= REGION3 720 91 REGION7 960 102 REGION10 <-- Selected 640 85





(Rev. 10/2004) Page 3-51

< Program output - continued > UNITS SECONDARY PRIMARY INPRIMARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP========================= ============= ============= ============= =====REGION6 116 1,240 3,100 3REGION4 123 1,320 3,410 3REGION8 118 1,300 3,170 3REGION10 85 640 2,320 3

NOTE: The above four lines make up file GRANTSPUOUT.TXT


The input for three-stage RHC can be greatly simplified if you only obtain information for each

selected primary unit (that is, Regions 4, 6, 8, and 10 here). The information consists of the size

of each secondary unit (university, here) and the number of third-stage units in the universe for

each secondary unit. This input is shown in files REGION4.TXT, REGION6.TXT,

REGION8.TXT, and REGION10.TXT. Each line in the files contains the number of third-stage

units (grants) in the universe and the size of that secondary unit (total grant dollars x 100,000), in

that order. After each of these four files is the computer output using the RHC Sample

Selection program. A sample of 10 universities is selected for each region. The results are:

REGION UNIVERSITIES 4 85, 46, 7, 82, 30, 34, 27, 66, 65, 80 6 113, 43, 78, 104, 89, 112, 30, 65, 3, 99 8 112, 6, 7, 93, 75, 111, 62, 115, 70, 99 10 78, 43, 7, 73, 55, 33, 10, 59, 64, 39

The previous five program runs (one at the primary level and four at the secondary level) created

five output files. Using a word processor or spreadsheet, these files can be joined to form one of

the input files (the one containing primary/secondary unit information) for the three-stage RHC

program which calculates the confidence interval. The file for this example is

PUSURHC3.TXT.


Page 3-52 (Rev. 10/2004)


(1) (2) (3)UNIV1 52 11UNIV2 37 9UNIV3 38 9UNIV4 20 5UNIV5 69 15UNIV6 69 15UNIV7 77 17UNIV8 32 7UNIV9 49 10UNIV10 73 15UNIV11 21 5UNIV12 62 13UNIV13 55 11UNIV14 59 12UNIV15 55 11UNIV16 36 8UNIV17 51 11UNIV18 26 7UNIV19 25 6UNIV20 73 15UNIV21 71 15UNIV22 47 10UNIV23 34 8UNIV24 25 6UNIV25 39 9UNIV26 49 10UNIV27 76 16UNIV28 21 5UNIV29 33 8UNIV30 54 11UNIV31 45 10UNIV32 74 16UNIV33 69 14UNIV34 50 10UNIV35 29 7UNIV36 56 12UNIV37 64 14UNIV38 66 14UNIV39 63 14UNIV40 57 12UNIV41 71 15UNIV42 45 10UNIV43 21 5UNIV44 46 10UNIV45 48 10UNIV46 44 9UNIV47 71 15UNIV48 67 14UNIV49 23 6UNIV50 54 11


UNIV101 34 8UNIV102 28 7UNIV103 73 15UNIV104 65 14UNIV105 68 14UNIV106 28 7UNIV107 55 11UNIV108 37 9UNIV109 54 11UNIV110 47 10UNIV111 44 9UNIV112 24 6UNIV113 50 10UNIV114 52 11UNIV115 66 14UNIV116 50 10UNIV117 66 14UNIV118 34 8UNIV119 73 16UNIV120 37 8UNIV121 42 9UNIV122 59 12UNIV123 45 11


Columns: (1) unit ID(2) number of grants


(Rev. 10/2004) Page 3-53

(3) size of university (grant amount x $100,000) DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/25/2004 GENERATION OF SECONDARY UNIT SAMPLE Time: 14:21 NAME OF INPUT FILE: C:\TEMP\REGION4.TXT


********* GROUP 1 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV57 16 76 UNIV48 14 67 UNIV35 7 29 UNIV107 11 55 UNIV85 <-- Selected 11 54 UNIV103 15 73 UNIV86 7 31 UNIV2 9 37 UNIV81 10 45 UNIV58 7 30 UNIV36 12 56 UNIV49 6 23


********* GROUP 2 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV52 11 52 UNIV6 15 69 UNIV46 <-- Selected 9 44 UNIV69 13 62 UNIV108 9 37 UNIV44 10 46 UNIV50 11 54 UNIV121 9 42 UNIV1 11 52 UNIV43 5 21 UNIV87 14 69 UNIV39 14 63



********* GROUP 10 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV53 11 56 UNIV24 6 25 UNIV42 10 45 UNIV120 8 37 UNIV105 14 68 UNIV97 12 56


Page 3-54 (Rev. 10/2004)

UNIV119 16 73 UNIV32 16 74 UNIV80 <-- Selected 15 70 UNIV96 7 29 UNIV13 11 55 UNIV62 13 61 UNIV59 16 75







(Rev. 10/2004) Page 3-55

UNIV1 56 10UNIV2 27 5UNIV3 56 11UNIV4 23 5UNIV5 72 13UNIV6 24 5UNIV7 61 11UNIV8 65 12UNIV9 68 13UNIV10 40 8UNIV11 64 12UNIV12 66 13UNIV13 80 14UNIV14 53 9UNIV15 36 7UNIV16 53 10UNIV17 47 9UNIV18 73 14UNIV19 41 8UNIV20 58 11UNIV21 45 9UNIV22 43 8UNIV23 56 10UNIV24 35 7UNIV25 34 7UNIV26 65 13UNIV27 78 14UNIV28 35 7UNIV29 31 6UNIV30 58 11UNIV31 29 6UNIV32 76 14UNIV33 57 10UNIV34 42 8UNIV35 69 13UNIV36 58 11UNIV37 31 6UNIV38 33 6UNIV39 40 8UNIV40 51 9UNIV41 60 11UNIV42 78 14UNIV43 39 7UNIV44 46 9UNIV45 58 11UNIV46 59 11UNIV47 53 10UNIV48 57 10UNIV49 28 6UNIV50 63 12UNIV51 31 6UNIV52 60 11UNIV53 30 6UNIV54 30 6UNIV55 40 8UNIV56 26 5UNIV57 24 5UNIV58 44 8

<-- continued -->UNIV59 67 13UNIV60 56 10UNIV61 33 7UNIV62 40 8UNIV63 68 13UNIV64 70 13UNIV65 57 10UNIV66 40 7UNIV67 54 10UNIV68 65 12UNIV69 62 12UNIV70 28 5UNIV71 56 10UNIV72 41 8UNIV73 31 6UNIV74 31 6UNIV75 46 9UNIV76 38 7UNIV77 62 12UNIV78 63 12UNIV79 50 9UNIV80 53 9UNIV81 39 7UNIV82 39 7UNIV83 39 7UNIV84 25 5UNIV85 67 13UNIV86 47 9UNIV87 54 10UNIV88 50 9UNIV89 35 7 UNIV90 66 13UNIV91 65 12UNIV92 71 13UNIV93 29 6UNIV94 74 14UNIV95 66 13UNIV96 71 13UNIV97 43 8UNIV98 62 11UNIV99 80 14UNIV100 57 11UNIV101 22 5UNIV102 33 6UNIV103 78 5UNIV104 25 9UNIV105 76 8UNIV106 39 5UNIV107 48 5UNIV108 54 7UNIV109 63 12UNIV110 28 8UNIV111 69 8UNIV112 27 10UNIV113 33 8UNIV114 52 7UNIV115 33 15UNIV116 23 10

Data set REGION6.TXT



Page 3-56 (Rev. 10/2004)

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/25/2004 GENERATION OF SECONDARY UNIT SAMPLE Time: 13:57 NAME OF INPUT FILE: C:\TEMP\REGION6.txt




********* GROUP 2 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV33 10 57 UNIV48 10 57 UNIV6 5 24 UNIV43 <-- Selected 7 39 UNIV68 12 65 UNIV41 11 60 UNIV46 11 59 UNIV1 10 56 UNIV40 9 51 UNIV88 9 50 UNIV36 11 58



********* GROUP 10 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV20 11 58 UNIV22 8 43 UNIV39 8 40 UNIV111 8 69 UNIV100 11 57 UNIV29 6 31 UNIV105 8 76 UNIV79 9 50 UNIV99 <-- Selected 14 80


(Rev. 10/2004) Page 3-57

UNIV13 14 80 UNIV60 10 56 UNIV54 6 30







Page 3-58 (Rev. 10/2004)




UNIV101 24 5UNIV102 26 6UNIV103 40 10UNIV104 77 16UNIV105 27 6UNIV106 65 15UNIV107 61 13UNIV108 36 9UNIV109 26 6UNIV110 38 9UNIV111 84 17UNIV112 75 16UNIV113 26 6UNIV114 45 10UNIV115 59 13UNIV116 59 13UNIV117 57 12UNIV118 58 12



(Rev. 10/2004) Page 3-59





********* GROUP 2 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV47 14 62 UNIV50 16 80 UNIV6 <-- Selected 8 34 UNIV44 15 71 UNIV68 16 75 UNIV42 10 49 UNIV48 11 54 UNIV1 15 72 UNIV41 9 36 UNIV89 6 26 UNIV37 7 31



********* GROUP 10 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV21 6 26 UNIV23 13 58 UNIV40 9 36 UNIV110 9 38 UNIV100 7 31 UNIV30 6 26 UNIV104 16 77 UNIV81 15 66 UNIV99 <-- Selected 13 60 UNIV13 8 33


Page 3-60 (Rev. 10/2004)

UNIV60 15 66 UNIV56 12 55







(Rev. 10/2004) Page 3-61

Data setREGION10.TXT

UNIV1 34 6UNIV2 32 5UNIV3 69 10UNIV4 23 4UNIV5 60 9UNIV6 72 11UNIV7 56 9UNIV8 28 5UNIV9 38 6UNIV10 60 9UNIV11 58 9UNIV12 37 6UNIV13 70 10UNIV14 37 6UNIV15 81 12UNIV16 53 9UNIV17 63 10UNIV18 32 5UNIV19 33 5UNIV20 37 6UNIV21 77 11UNIV22 52 8UNIV23 63 10UNIV24 41 7UNIV25 45 8UNIV26 34 6UNIV27 61 10UNIV28 70 10UNIV29 34 5UNIV30 22 4UNIV31 66 10UNIV32 69 10UNIV33 65 10UNIV34 26 4UNIV35 43 7UNIV36 65 10UNIV37 80 12UNIV38 74 11UNIV39 38 6UNIV40 43 7UNIV41 47 8UNIV42 59 9UNIV43 42 7UNIV44 54 9UNIV45 73 11

<--continued -->

UNIV46 78 12UNIV47 72 11UNIV48 30 5UNIV49 47 8UNIV50 52 8UNIV51 24 4UNIV52 26 4UNIV53 22 4UNIV54 57 9UNIV55 78 12UNIV56 62 10UNIV57 57 9UNIV58 68 10UNIV59 52 8UNIV60 54 9UNIV61 41 7UNIV62 61 10UNIV63 79 12UNIV64 50 8UNIV65 54 9UNIV66 53 9UNIV67 40 7UNIV68 44 7UNIV69 39 7UNIV70 72 11UNIV71 76 11UNIV72 34 5UNIV73 27 5UNIV74 40 7UNIV75 41 7UNIV76 25 4UNIV77 41 7UNIV78 39 7UNIV79 58 9UNIV80 71 11UNIV81 37 6UNIV82 30 5UNIV83 78 12UNIV84 59 9UNIV85 29 5

Note: This file has 85 lines.


Page 3-62 (Rev. 10/2004)



********* GROUP 1 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV44 9 54 UNIV32 10 69 UNIV77 7 41 UNIV78 <-- Selected 7 39 UNIV2 5 32 UNIV50 8 52 UNIV34 4 26 UNIV46 12 78


********* GROUP 2 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV6 11 72 UNIV43 <-- Selected 7 42 UNIV62 10 61 UNIV41 8 47 UNIV1 6 34 UNIV40 7 43 UNIV79 9 58 UNIV36 10 65



********* GROUP 10 ********* SECONDARY UNIT 3RD STAGE SECONDARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= UNIV71 11 76 UNIV9 6 38 UNIV21 11 77 UNIV23 10 63 UNIV39 <-- Selected 6 38 UNIV29 5 34 UNIV72 5 34 UNIV13 10 70 UNIV51 4 24



(Rev. 10/2004) Page 3-63





Constructing the data file

The data file for this example (PUSURHC3.TXT) is shown on the next page. This file was

constructed using the RHC Sample Selection program to select the primary units (regions) and

within each selected primary unit, the 10 secondary units (universities). The four lines beginning

with REGIONx are from the output file created during the primary unit selection

(GRANTSPUOUT.TXT). The 10 lines after each REGIONx line consist of the output file

created when selecting the universities from each region (OUTREGION4.TXT, . . .,

OUTREGION10.TXT). Using a word processor or spreadsheet, a column containing the sample

sizes (highlighted) must be added to the files created by the five RHC Sample Selection

programs.


Page 3-64 (Rev. 10/2004)

--- Data set PUSURHC3.TXT ---

REGION4 123 10 1320 3410 3UNIV85 54 11 11 125 12UNIV46 44 9 9 131 12UNIV7 77 15 17 119 12UNIV82 52 10 11 129 12UNIV30 54 11 11 141 12UNIV34 50 10 10 140 12UNIV27 76 15 16 138 12UNIV66 76 15 16 128 13UNIV65 62 12 14 125 13UNIV80 70 14 15 155 13REGION6 116 10 1240 3100 3UNIV113 33 7 8 108 11UNIV43 39 8 7 105 11UNIV78 63 13 12 104 11UNIV104 25 5 9 96 11UNIV89 35 7 7 124 12UNIV112 27 5 10 108 12UNIV30 58 12 11 95 12UNIV65 57 11 10 109 12UNIV3 56 11 11 115 12UNIV99 80 16 14 113 12REGION8 118 10 1300 3170 3UNIV112 75 15 16 125 11UNIV6 34 7 8 127 11UNIV7 51 10 11 120 12UNIV93 54 11 11 136 12UNIV75 52 10 11 126 12UNIV111 84 17 17 134 12UNIV62 64 13 14 123 12UNIV115 59 12 13 137 12UNIV70 65 13 14 143 12UNIV99 60 12 13 129 12REGION10 85 10 640 2320 3UNIV78 39 8 7 62 8UNIV43 42 8 7 68 8UNIV7 56 11 9 54 8UNIV73 27 5 5 63 8UNIV55 78 16 12 70 8UNIV33 65 13 10 77 9UNIV10 60 12 9 76 9UNIV59 52 10 8 71 9UNIV64 50 10 8 73 9UNIV39 38 8 6 68 9


(Rev. 10/2004) Page 3-65


Since approximately 20% of the grants at each selected university are to be audited, the

following sample sizes are determined:






Page 3-66 (Rev. 10/2004)

The data file containing the errors for these 438 audited grants is DATARHC3.TXT, shown on

the next page. The values for the first two universities in Region 4 and the last two universities

in Region 10 are illustrated. Each sample value is equal to the total charges after the scheduled

completion of the grant (in thousands of dollars). Notice that each line begins with a counter.

Finally, the RHC Three-Stage program is run, which generates a confidence interval for the

universe total using input files PUSURHC3.TXT and DATARHC3.TXT. The output from this

program is shown at the end of this section.


(Rev. 10/2004) Page 3-67

--- Data set DATARHC3.TXT ---

1 82 03 64 65 06 137 18 79 210 1311 1312 4 These are the sample values for UNIV85 and UNIV46 in Region 4.13 614 015 1516 1217 918 019 1320 10...421 0422 6423 19424 17425 13426 12427 13 These are the sample values for UNIV64 and UNIV39 in Region 10.428 12429 11430 14431 13432 0433 1434 5435 16 436 0437 0438 8


Page 3-68 (Rev. 10/2004)

Summary of results. Referring to the last page in the computer output, the estimate of the

universe total (all 12 regions) is the OVERALL POINT ESTIMATE of 463,526 ($463,526,000)

with a corresponding estimated OVERALL STANDARD ERROR of 53,521 ($53,521,000).

NOTE: This estimate does not require knowing the number of grants in the universe.

The program also provides estimates for the total error amount for each sampled P.U. (region)

and for each of the groups of S.U.s (universities) within each sampled region. For example, the

estimated error amount for Region 4 is 50,529 ($50,529,000) and the estimated error amount for

the group of 12 universities containing UNIV85 is 3,849 ($3,849,000). The SIZES RATIO

refers to the ratio of the size of the group containing this university to the size of this university.

To illustrate, UNIV85 in Region 4 has a size of 11 and is in a group of size 125 (look at file

REGION4.TXT). The SIZES RATIO here is 125/11 = 11.3636.

The 80% confidence interval for the total error amount is 394,936 to 532,116 ($394,936,000 to

$532,116,000). The PRECISION AMOUNT is the amount added and subtracted to the point

estimate (463,526) to obtain the corresponding confidence interval. In the 80% confidence

interval, the lower limit of 394,936 is obtained by subtracting the precision amount of 68,590

from 463,526. The PRECISION PERCENT is the precision amount divided by the point

estimate, expressed as a percentage.


(Rev. 10/2004) Page 3-69

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/26/2004 RHC THREE-STAGE VARIABLE APPRAISAL Time: 16:27 AUDIT/REVIEW: RHC 3-Stage

DATA FILE USED: C:\TEMP\DATARHC3.TXT PRIMARY/SECONDARY UNIVERSE FILE USED: C:\TEMP\PUSURHC3.txt

*****D I F F E R E N C E***** **** SAMPLED UNITS **** THIRD STAGE SAMPLE SAMPLE NONZEROPRIMARY / SECONDARY IDENTIFICATION UNIVERSE SIZE VALUE COUNT================================== =========== ====== ============ ========REGION4 UNIV85 54 11 69.00 9 UNIV46 44 9 69.00 7 UNIV7 77 15 145.00 14 UNIV82 52 10 91.00 10 UNIV30 54 11 83.00 9 UNIV34 50 10 69.00 8 UNIV27 76 15 72.00 12 UNIV66 76 15 134.00 13 UNIV65 62 12 97.00 10 UNIV80 70 14 153.00 11 Total 615 122 103REGION6 UNIV113 33 7 81.00 7 UNIV43 39 8 36.00 8 UNIV78 63 13 83.00 9 UNIV104 25 5 56.00 5 UNIV89 35 7 82.00 6 UNIV112 27 5 29.00 2 UNIV30 58 12 65.00 9 UNIV65 57 11 82.00 8 UNIV3 56 11 67.00 6 UNIV99 80 16 164.00 15 Total 473 95 75REGION8 UNIV112 75 15 78.00 12 UNIV6 34 7 60.00 7 UNIV7 51 10 48.00 9 UNIV93 54 11 85.00 9 UNIV75 52 10 63.00 7 UNIV111 84 17 154.00 16 UNIV62 64 13 83.00 13 UNIV115 59 12 61.00 8 UNIV70 65 13 87.00 9 UNIV99 60 12 90.00 9 Total 598 120 99REGION10 UNIV78 39 8 75.00 7 UNIV43 42 8 71.00 6 UNIV7 56 11 123.00 11 UNIV73 27 5 32.00 3 UNIV55 78 16 123.00 11 UNIV33 65 13 113.00 11 UNIV10 60 12 104.00 8 UNIV59 52 10 77.00 7 UNIV64 50 10 117.00 9 UNIV39 38 8 43.00 5 Total 507 101 78


Page 3-70 (Rev. 10/2004)

*****D I F F E R E N C E***** **** SAMPLED UNITS **** THIRD STAGE SAMPLE SAMPLE NONZEROPRIMARY / SECONDARY IDENTIFICATION UNIVERSE SIZE VALUE COUNT================================== =========== ====== ============ ========

TOTALS 2,193 438 3,414.00 355

--- POINT ESTIMATES--- *****D I F F E R E N C E***** **** SAMPLED UNITS **** SIZES POINT PRIMARY / SECONDARY IDENTIFICATION SAMPLE MEAN RATIO ESTIMATE ================================== =========== =========== =============REGION4 UNIV85 Note: 3,849 is the estimate 6.27 11.3636 3,849 UNIV46 for the group containing 7.67 14.5556 4,910 UNIV7 UNIV85 (not just UNIV85). 9.67 7.0000 5,210 UNIV82 This group contained 12 9.10 11.7273 5,549 UNIV30 universities show earlier 7.55 12.8182 5,223 UNIV34 in the output using data 6.90 14.0000 4,830 UNIV27 set REGION4.TXT. 4.80 8.6250 3,146 UNIV66 8.93 8.0000 5,431 UNIV65 8.08 8.9286 4,475 UNIV80 10.93 10.3333 7,905

TOTAL Estimate for Region 4 ý 50,529

REGION6 UNIV113 11.57 13.5000 5,155 UNIV43 4.50 15.0000 2,633 UNIV78 6.38 8.6667 3,486 UNIV104 11.20 10.6667 2,987 UNIV89 11.71 17.7143 7,263 UNIV112 5.80 10.8000 1,691 UNIV30 5.42 8.6364 2,713 UNIV65 7.45 10.9000 4,632 UNIV3 6.09 10.4545 3,566 UNIV99 10.25 8.0714 6,619

TOTAL 40,744


TOTAL 40,913

--- POINT ESTIMATES--- *****D I F F E R E N C E***** **** SAMPLED UNITS **** SIZES POINT PRIMARY / SECONDARY IDENTIFICATION SAMPLE MEAN RATIO ESTIMATE ================================== =========== =========== =============REGION10 UNIV78 9.38 8.8571 3,238 UNIV43 8.88 9.7143 3,621 UNIV7 11.18 6.0000 3,757 UNIV73 6.40 12.6000 2,177 UNIV55 7.69 5.8333 3,498


(Rev. 10/2004) Page 3-71

UNIV33 8.69 7.7000 4,351 UNIV10 8.67 8.4444 4,391 UNIV59 7.70 8.8750 3,554 UNIV64 11.70 9.1250 5,338 UNIV39 5.38 11.3333 2,315

TOTAL 36,240


**** SAMPLED UNITS **** WITHIN BETWEEN TOTALPRIMARY UNIT IDENTIFICATION VARIANCE VARIANCE VARIANCE============================== ============== ============== ==============REGION4 757,373 10,383,517 11,140,890REGION6 798,923 29,209,394 30,008,317REGION8 710,631 8,065,650 8,776,280REGION10 781,064 6,460,707 7,241,771

(Values of V4) (Values of V3)

--- OVERALL VARIANCE COMPONENTS ---


2,713,022,822 151,453,466 2,864,476,288

(Value of V1) (Value of V2)

*****D I F F E R E N C E***** --- SUMMARY OF APPRAISAL RESULTS ---PRIMARY UNITS SAMPLED 4PRIMARY UNITS NOT SAMPLED 8TOTAL PRIMARY UNITS 12OVERALL POINT ESTIMATE 463,526OVERALL STANDARD ERROR 53,521





Page 3-72 (Rev. 10/2004)

Discussion. In general, you can expect greater precision with the RHC procedure, provided

there is a significant correlation between NUMBER OF UNITS and SIZE OF UNIT. To

illustrate, consider the file containing the primary unit information used in the three-stage RHC

illustration:

(A) (B) REGION1 117 1250 REGION2 63 610 REGION3 91 720 REGION4 123 1320 REGION5 107 1160 REGION6 116 1240 REGION7 102 960 REGION8 118 1300 REGION9 122 1320 REGION10 85 640 REGION11 94 930 REGION12 62 550

Columns: (A) number of universities (S.U.s)(B) size of each P.U. (grant amount x $100,000)

For this example, the correlation between columns (A) and (B) is .958, and we would expect a

single- and two-stage RHC procedure to work quite well. For a three-stage procedure, this

correlation rule must also apply within the sampled primary units, at the secondary unit level.

As mentioned earlier, the benefits of RHC sampling include the following:

• Precision is increased if the above correlation rule is satisfied.

• The flavor of pps sampling is maintained, since pps sampling is used to select a unit fromeach random group.

• Computations are relatively simple and straightforward.

• The point estimate is stable. This implies that when sampling indefinitely, the point( $)Testimate will exhibit relatively small variation.


(Rev. 10/2004) Page 3-73

• The point estimate of the variance of is stable, producing more reliable confidence$Tintervals.

This implies that when sampling indefinitely, the lower confidence limits will exhibit relatively

small variation.


Page 3-74 (Rev. 10/2004)

FORMULAS

Definitions

1. Si = (size of i-th P.U.)/(size of entire population)

2. Bi = ESi over the i-th group of P.U.s

3. Sij = (size of j-th S.U. in the i-th sampled P.U.)/(size of i-th sampled P.U.)

(Note: denominator of Sij = numerator of Si)

4. Bij = ESij over the j-th group in i-th sampled P.U.

5. N = number of P.U.s (population)

6. n = number of P.U.s (sample)

7. Mi = number of S.U.s in i-th sampled P.U. (population)

8. mi = number of S.U.s in i-th sampled P.U. (sample)

9. Kij = number of third-stage units for j-th sampled S.U. in i-th sampled P.U. (population)

10. kij = number of third-stage units for j-th sampled S.U. in i-th sampled P.U. (sample)


$$

TTSi

i

ii

n

=⎛

⎝⎜

⎞

⎠⎟

=∑π

1

where = estimator of total for i-th sampled P.U.$Ti

= (equation 1)π ijij

ijj

m TS

i $⎛

⎝⎜⎜

⎞

⎠⎟⎟

=∑

1

and = estimator of population total for j-th sampled S.U. in i-th sampled P.U.$Tij

= where = average of kij units at the third stageK yij ij yij

NOTE: It can be shown that is an unbiased estimator of T.$T


(Rev. 10/2004) Page 3-75


V1 + V2 wherev T( $) =

(equation 2)V

N N

N N

TS

Ti

i

n

ii

n ii

ii

n

1

2

1

2 2

1

2

1=

−

−

−⎛

⎝⎜

⎞

⎠⎟=

=

=

∑

∑∑π

$$

and

(equation 3)VS

v Ti

ii

n

i21

=⎛⎝⎜

⎞⎠⎟

=∑ π

( $ )

and Ni = number of P.U.s in the i-th group after the random split into n groups.

is obtained by applying the two-stage RHC procedure within the i-th sampled P.U.;v Ti( $ )

i.e., the i-th P.U. is viewed as the entire population. Consequently,

= V3,i + V4,iv Ti( $ )

where

V

M M

M M

K yS

Ti

ij ij

m

i ijj

m ijij ij

iji

j

m

i

i

i

3

2

1

2 2

1

2

1, $=

−

−

−⎛

⎝⎜⎜

⎞

⎠⎟⎟

=

=

=

∑

∑∑π

and


Page 3-76 (Rev. 10/2004)

VKS

K kk

si ijij

ijj

mij ij

ijij

i

41

2, =

−⎛

⎝⎜⎜

⎞

⎠⎟⎟

=∑π

and where

(1) Mij = the number of S.U.s in the j-th random group within the i-th sampled P.U.

(2) = average of the kij items for the j-th sampled S.U. within the i-th sampled P.U.yij

Comments 1. V1 is essentially the same expression obtained for the single-stage RHC

procedure and will be referred to as the “between unit” variation.

2. V2 is the contribution of the 2nd- and 3rd-stage variation and is obtained by

treating each sampled P.U. as the population to be sampled using two

(additional) stages.

3. The estimated standard error of is .$T v T( $)


$ . ( $)T v T± 1959963984540


RAT-STATS Companion Manual Stratified Cluster Variable Appraisal

(Rev. 10/2004) Page 3-77

Stratified Cluster Variable Appraisal

With this procedure, you first stratify, then obtain a cluster (single-stage) sample within each

stratum. This is motivated by the discussion in the RAT-STATS User’s Guide.

The estimate of a universe total is the sum of the estimates for each stratum. The estimated

variance of this estimator is the sum of the estimated variances for each stratum.

Example 7. In a large section of the U.S., an audit was conducted for 583 universities with

health related research grants. It was decided to define two strata:

Stratum 1: state-supported universities (N1 = 415)

Stratum 2: private universities (N2 = 168)

Within each stratum, a single-stage cluster sample was obtained with n1 = 25 universities

selected from Stratum 1 and n2 = 10 universities from Stratum 2. For each of the sampled

universities, all health-related grants would be audited (since there weren't that many at each

university) to determine the amount of charges improperly charged to these grants. The

following data were obtained, where yj is the total of the improper charges (in thousands of

dollars) for the j-th university (cluster) and Mj is the number of grants (universe) for this

university, all of which are audited.

NOTE: The number of grants audited at each university (the Mi values) are not used in theprogram calculations. They are supplied for informational purposes only. Forexample, if all the Mi values are set equal to 1, the resulting confidence intervals willbe unchanged.

Stratified Cluster Variable Appraisal RAT-STATS Companion Manual

Page 3-78 (Rev. 10/2004)

Stratum 1

Univ. Mj yj Univ. Mj yj 1 8 96 14 10 49 2 12 121 15 9 53 3 4 42 16 3 50 4 5 65 17 6 32 5 6 52 18 5 22 6 6 40 19 5 45 EMj = 151 7 7 75 20 4 37 8 5 65 21 6 51 Eyj = 1,329 9 8 45 22 8 30 10 3 50 23 7 39 11 2 85 24 3 47 12 6 43 25 8 41 13 5 54

Stratum 2

Univ. Mj yj Univ. Mj yj 1 2 18 6 8 96 2 5 52 7 6 64 EMj = 49 3 7 68 8 10 115 4 4 36 9 3 41 Eyj = 547 5 3 45 10 1 12

These values were stored in data file DATASTRCLUS.TXT. Immediately following the listing

of this data file is the resulting computer output using the VARIABLE STRATIFIED CLUSTER

program.

Data file DATASTRCLUS.TXTSTATE UNIVERSITIES 415 25UNIV1 8 96UNIV2 12 121UNIV3 4 42UNIV4 5 65UNIV5 6 52UNIV6 6 40UNIV7 7 75UNIV8 5 65UNIV9 8 45UNIV10 3 50UNIV11 2 85UNIV12 6 43UNIV13 5 54UNIV14 10 49


(Rev. 10/2004) Page 3-79

UNIV15 9 53UNIV16 3 50UNIV17 6 32UNIV18 5 22UNIV19 5 45UNIV20 4 37UNIV21 6 51UNIV22 8 30UNIV23 7 39UNIV24 3 47UNIV25 8 41PRIVATE UNIVERSITIES 168 10UNIV1 2 18UNIV2 5 52UNIV3 7 68UNIV4 4 36UNIV5 3 45UNIV6 8 96UNIV7 6 64UNIV8 10 115UNIV9 3 41UNIV10 1 12

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/23/2004 STRATIFIED CLUSTER VARIABLE APPRAISAL Time: 14:36 AUDIT/REVIEW: Variable - Stratified Cluster

DATA FILE USED: C:\Temp\DATASTRCLUS.TXT

STRATUM IDENTIFICATION SAMPLE SAMPLE CLUSTER IDENTIFICATION UNIVERSE SIZE SAMPLED VALUE POINT ESTIMATE=========================== =========== ====== ============= ==============STATE UNIVERSITIES 415 25 UNIV1 8 8 96.00 UNIV2 12 12 121.00 UNIV3 4 4 42.00 UNIV4 5 5 65.00 UNIV5 6 6 52.00 UNIV6 6 6 40.00 UNIV7 7 7 75.00 UNIV8 5 5 65.00 UNIV9 8 8 45.00 UNIV10 3 3 50.00 UNIV11 2 2 85.00 UNIV12 6 6 43.00 UNIV13 5 5 54.00 UNIV14 10 10 49.00 UNIV15 9 9 53.00 UNIV16 3 3 50.00 UNIV17 6 6 32.00 UNIV18 5 5 22.00 UNIV19 5 5 45.00 UNIV20 4 4 37.00 UNIV21 6 6 51.00 UNIV22 8 8 30.00

Stratified Cluster Variable Appraisal RAT-STATS Companion Manual

Page 3-80 (Rev. 10/2004)

UNIV23 7 7 39.00 UNIV24 3 3 47.00 UNIV25 8 8 41.00 STRATUM TOTALS 151 151 1,329.00 22,061

PRIVATE UNIVERSITIES 168 10 UNIV1 2 2 18.00 UNIV2 5 5 52.00 UNIV3 7 7 68.00 UNIV4 4 4 36.00 UNIV5 3 3 45.00 UNIV6 8 8 96.00 UNIV7 6 6 64.00 UNIV8 10 10 115.00 UNIV9 3 3 41.00 UNIV10 1 1 12.00 STRATUM TOTALS 49 49 547.00 9,190

STRATUM IDENTIFICATION SAMPLE SAMPLE CLUSTER IDENTIFICATION UNIVERSE SIZE SAMPLED VALUE POINT ESTIMATE=========================== =========== ====== ============= ==============

STRATA TOTALS 583 35CLUSTER UNIT TOTALS 200 200 1,876.00OVERALL POINT ESTIMATE 31,251OVERALL STANDARD ERROR 2,418

CONFIDENCE LEVEL ---80 PERCENT-- ---90 PERCENT-- ---95 PERCENT--LOWER LIMIT 28,152 27,273 26,511UPPER LIMIT 34,350 35,229 35,991PRECISION AMOUNT 3,099 3,978 4,740PRECISION PERCENT 9.92% 12.73% 15.17%Z-VALUE USED 1.281551565545 1.644853626951 1.959963984540

Discussion. For stratum 1, the unbiased estimate of the universe total is

= (415/25)(1,329) = 22,061 ($22,061,000)$T1

The unbiased estimate of the universe total for stratum 2 is

= (168/10)(547) = 9,190 ($9,190,000)$T2

Consequently, an unbiased estimate of the universe total (highlighted) is

= 31,251 ($31,251,000)$ $ $T T T= +1 2

Using formula 2, the estimated variance of is = 5,848,565 and the$T v T v T v T( $) ( $ ) ( $ )= +1 2

corresponding standard error (highlighted) is 2,418.

The approximate 95% confidence interval for the universe total is


(Rev. 10/2004) Page 3-81

$ . ( $)T v T±1959963984540

31,251 ± 1.959963984540(2418)

that is, 26,511 to 35,991 ($26,511,000 to $35,991,000).

FORMULAS

1. Estimated total in the universe (T)

$,T

Nn

y N yh

hh

L

j hj

n

h hh

Lh

=⎛

⎝⎜

⎞

⎠⎟ =

= = =∑ ∑ ∑

1 1 1


Nh = number of clusters (universe) for stratum h

nh = number of clusters (sample) for stratum h

yj,h = total of the variable of interest (e.g., errors) for the j-th P.U. within stratum h

= sample average for stratum hyh

NOTE: Let = estimated total for stratum h. Then = and $Th$Th N yh h

$ $T Th= ∑


v T N N nn n

y y v Th h h

h hh

L

j h hj

n

hh

Lh

( $) ( )( )

( ) ( $ ),=−−

− == = =∑ ∑ ∑11

2

1 1



Stratified Multistage Variable Appraisal RAT-STATS Companion Manual

Page 3-82 (Rev. 10/2004)

Stratified Multistage Variable Appraisal

As with the stratified cluster procedure, you must first stratify the universe. Rather than take a

cluster (single-stage) sample within each stratum, you will obtain a multistage (two-stage or

three-stage) sample within each stratum. These multistage samples may be random (using the

Two-Stage Unrestricted or Three-Stage Unrestricted programs) or may be obtained using the

RHC procedure and the RHC Two-Stage or RHC Three-Stage programs.

Unlike the Stratified Cluster program, this program requires that you first run the appropriate

multistage program on each stratum and record the results. The output results are then used as

input to the Stratified Multistage program. You may store the results from each stratum (point

estimate, standard error) in a file or simply input these values interactively.

NOTE: The “universe size” refers to the number of units at the most detailed level of themultistage sample. For example, if you are obtaining a three-stage sample withineach stratum, then the “universe size” refers to the total number of third-stage unitswithin this stratum.

Example 8. This example is similar to Example 7 in the Stratified Cluster section. In a

particular region, the universe consisting of university grants is stratified by defining

Stratum 1: state-supported universities (N1 = 120 univ.)

and Stratum 2: private universities (N2 = 85 univ.)

Because these universities are so widespread, it was decided to employ a two-stage sample using

15 state supported universities and 10 private universities. Rather than audit all grants at a

selected university, it was decided (based on available resources) to audit roughly 20% of the

grants at each selected university to estimate the amount of charges improperly charged to these

RAT-STATS Companion Manual Stratified Multistage Variable Appraisal

(Rev. 10/2004) Page 3-83

grants. We know that there are a total of 5,800 grants within the universe of the 120 state

supported universities and 4,500 grants within the 85 private universities.

The following data were obtained where yi,j is the dollars (in thousands) of improper charges for

the j-th grant within the i-th sampled university, Mi is the total number of grants at the i-th

university, and mi is the number of audited grants at the i-th university. Also, and are theyi si2

mean and variance of the sample values from the i-th university.

NOTE: The 15 state-supported universities and 10 private universities were obtained usingthe Single-Stage Random Numbers program. For ease of illustration, they will bereferred to as University 1, 2, 3, . . . within each stratum.

The corresponding data files are the input files for the Two-Stage Unrestricted program. These

are files STRMULT1.TXT and STRMULT2.TXT. The files containing the universe sizes are

UNIV1.TXT and UNIV2.TXT.


Page 3-84 (Rev. 10/2004)

State-supported universities

Univ. Mi mi Dollars (yi,j,in thousands) y-i si

2

1 60 12 4 0 0 6 7 11 0 5 4 0 8 2 3.92 13.36 2 50 10 4 7 0 0 6 10 3 3 2 0 3.50 11.17 3 45 9 3 1 0 5 5 8 10 0 4 4.00 12.00 4 40 8 2 7 10 0 6 5 4 0 4.25 12.21 5 55 11 7 1 6 0 0 0 5 12 8 2 4 4.09 15.49 6 58 12 0 0 6 5 12 8 2 0 7 7 2 5 4.50 14.27 7 62 12 4 1 0 3 10 7 6 6 0 5 4 4 4.17 8.70 8 52 10 3 8 0 6 2 10 0 0 5 2 3.60 12.49 9 50 10 7 1 4 0 2 6 0 0 0 5 2.50 7.61 10 45 9 2 1 0 0 0 5 8 6 10 3.56 14.53 11 40 8 4 7 3 8 0 1 6 10 4.88 12.13 12 48 10 0 0 6 5 2 3 8 0 12 0 3.60 16.93 13 57 11 3 0 1 10 5 0 6 6 7 3 8 4.45 11.07 14 60 12 4 1 0 0 0 6 8 10 4 3 2 5 3.58 10.63 15 54 11 3 0 1 1 4 8 6 9 5 0 3 3.64 9.65

Private universities

Univ. Mi mi Dollars (yi,j,in thousands) y-i si

2

1 66 13 4 4 0 0 0 3 6 7 5 2 0 0 4 2.69 6.40 2 52 10 10 1 6 4 0 0 5 8 12 7 5.30 17.12 3 47 9 2 8 1 0 6 6 11 5 0 4.33 14.75 4 55 11 3 8 0 0 6 5 2 1 12 8 5 4.55 14.47 5 48 10 0 5 8 1 1 6 10 7 9 3 5.00 12.89 6 60 12 7 3 0 5 6 6 0 0 8 2 3 5 3.75 8.02 7 57 11 2 4 0 6 10 7 5 0 0 3 8 4.09 11.89 8 50 10 3 5 1 1 0 0 3 8 10 7 3.80 12.62 9 62 12 8 4 0 2 2 6 5 1 12 5 0 0 3.75 13.66 10 56 11 5 0 1 2 8 7 10 6 0 4 2 4.09 11.49

The data files are shown on the next page. The corresponding computer outputs immediately

follow.


(Rev. 10/2004) Page 3-85

Data file STRMULT1.TXT Data file STRMULT2.TXT 1 4 1 4 2 0 2 4 3 0 3 0 4 6 4 0 5 7 5 0 6 11 6 3 7 0 7 6 8 5 8 7 9 4 9 5 10 0 10 2 11 8 11 0 12 2 12 0 13 4 13 4 14 7 14 10 15 0 15 1 16 0 16 6 17 6 17 4 18 10 18 0 19 3 19 0 20 3 20 5 21 2 21 8 22 0 22 12 . 23 7 . . . .133 4 .134 1 87 8135 0 88 4136 0 89 0137 0 90 2138 6 91 2139 8 92 6140 10 93 5141 4 94 1142 3 95 12143 2 96 5144 5 97 0145 3 98 0146 0 99 5147 1 100 0148 1 101 1149 4 102 2150 8 103 8151 6 104 7152 9 105 10153 5 106 6154 0 107 0155 3 108 4 109 2


Page 3-86 (Rev. 10/2004)

Data file UNIV1.TXT Data file UNIV2.TXT 1 60 12 1 66 13 2 50 10 2 52 10 3 45 9 3 47 9 4 40 8 4 55 11 5 55 11 5 48 10 6 58 12 6 60 12 7 62 12 7 57 11 8 52 10 8 50 10 9 50 10 9 62 12 10 45 9 10 56 11 11 40 8 12 48 10 13 57 11 14 60 12 15 54 11

Output using the state-supported universities

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 TWO-STAGE UNRESTRICTED VARIABLE APPRAISAL Time: 14:14 AUDIT/REVIEW: Stratum 1

DATA FILE USED: C:\TEMP\STRMULT1.TXT


1 12/8 3.92 13.36 60 235 2 10/7 3.50 11.17 50 175 3 9/7 4.00 12.00 45 180 4 8/6 4.25 12.21 40 170 5 11/8 4.09 15.49 55 225 6 12/9 4.50 14.27 58 261 7 12/10 4.17 8.70 62 258 8 10/7 3.60 12.49 52 187 9 10/6 2.50 7.61 50 125 10 9/6 3.56 14.53 45 160 11 8/7 4.88 12.12 40 195 12 10/6 3.60 16.93 48 173 13 11/9 4.45 11.07 57 254 14 12/9 3.58 10.63 60 215 15 11/9 3.64 9.65 54 196

155/114 4.15 776 3,010


STANDARD ERROR 1,277


(Rev. 10/2004) Page 3-87




Output using the private universities

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 TWO-STAGE UNRESTRICTED VARIABLE APPRAISAL Time: 14:13 AUDIT/REVIEW: Stratum 2

DATA FILE USED: C:\TEMP\STRMULT2.TXT


1 13/8 2.69 6.40 66 178 2 10/8 5.30 17.12 52 276 3 9/7 4.33 14.75 47 204 4 11/9 4.55 14.47 55 250 5 10/9 5.00 12.89 48 240 6 12/9 3.75 8.02 60 225 7 11/8 4.09 11.89 57 233 8 10/8 3.80 12.62 50 190 9 12/9 3.75 13.66 62 232 10 11/9 4.09 11.49 56 229

109/84 4.26 553 2,257


STANDARD ERROR 873


Page 3-88 (Rev. 10/2004)




When running the STRATIFIED MULTISTAGE program, you should see the following input

window containing values for the first stratum. The program output immediately follows.


(Rev. 10/2004) Page 3-89

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 STRATIFIED MULTISTAGE VARIABLE APPRAISAL Time: 14:41 AUDIT/REVIEW: Combining the Strata

THE ESTIMATORS ARE BASED ON THE FOLLOWING ENTRIES: STRATUM POINT ESTIMATE STANDARD ERROR 1 24,077 1,277 2 19,182 873

= = = = = = = = = = = = = = = = = = RESULTS = = = = = = = = = = = = = = = = = =

POINT ESTIMATE STANDARD ERROR

43,259 1,547


Discussion. The point estimate for the universe total is the sum of the point estimates for the

two strata; that is, = 24,077 + 19,182 = 43,259 ($43,259,000). The estimated variance of $T $T

is = (1277)2 + (873)2 = 2,392,858 and the corresponding standard error is v T( $) 2 392 858, ,

= 1,547. The approximate 95% confidence interval for the universe total is

43,259 ± 1.959963984540(1,547)

that is, 40,227 to 46,291 ($40,227,000 to $46,291,000).


Page 3-90 (Rev. 10/2004)

$ . ( $)T v T±1959963984540

FORMULAS

1. Estimated total in the universe (T)

$ $T Thh

L

==∑

1

where is the point estimate for the universe total in stratum h and L is the number of$Th

strata.


v T v Thh

L

( $) ( $ )==∑

1

where is the estimated variance of and is equal to the square of the standard error of v Th( $ ) $Th

.$Th



RAT-STATS Companion Manual Poststratification

(Rev. 10/2004) Page 3-91

Poststratification

Oftentimes sampling problems arise in which the user would like to stratify on a key variable but

cannot place the sampling units into their correct strata until after the sample is selected.

Another situation arises when an auditor does not recognize a need to stratify prior to obtaining a

simple random sample and the sample items are evaluated. The poststratification program is

designed for such situations and provides reliable results if the overall sample size is large and

the poststratified sample sizes are large (say, at least 20). It is however, less efficient than using

a prestratified sample; that is, produces a slightly wider confidence interval for the same sample

size.

A key thing to keep in mind here is that the universe strata sizes must be known.

Consequently, before you define a set of strata, make sure that the you know the number of

universe items in each of the strata. The program does not allow you to estimate these universe

sizes. Poststratification is often appropriate when a simple random sample is not properly

balanced according to major groupings of the population.

Example 9. In a recent hospital audit, the amount of unallowable bad debts was determined for

a particular year. For the universe, it was known that there were N1 = 373 inpatient bad debts

(Stratum 1) and N2 = 1,146 outpatient bad debts (Stratum 2). The total universe size is N = N1 +

N2 = 1,519. So there are roughly 20% inpatient and 80% outpatient bad debts in the universe.

Suppose that a simple random sample of 100 bad debts revealed:

Poststratification RAT-STATS Companion Manual

Page 3-92 (Rev. 10/2004)

Inpatient bad debts Outpatient bad debts n1 = 45 n2 = 55

= $240.00 = $30.00y1 y2

= 22.04 = 198.56s12 s2

2

The data files (POSTDATA.TXT and UNIVPOST.TXT) are shown on the next page and the

resulting computer output immediately follows.


(Rev. 10/2004) Page 3-93

Data file POSTDATA.TXT < - continued - > Universe file UNIVPOST.TXT

1 242.27 2 240.43 3 243.43 4 241.00 5 235.71 6 239.74 7 249.53 8 232.61 9 243.51 10 243.70 11 241.36 12 238.86 13 241.20 14 246.14 15 236.91 16 244.91 17 231.91 18 249.58 19 227.84 20 239.07 21 232.89 22 242.15 23 238.93 24 243.41 25 241.61 26 240.04 27 229.43 28 238.12 29 240.82 30 241.84 31 236.08 32 248.11 33 239.39 34 236.00 35 238.67 36 242.88 37 241.90 38 238.96 39 234.03 40 237.86 41 239.01 42 239.21 43 245.45 44 242.08 45 241.39 46 46.32 47 37.59 48 31.23 49 22.92 50 44.96

51 25.9952 31.7553 42.3754 28.8255 14.8056 22.4457 12.9358 37.2259 12.2960 21.0261 24.7662 49.6963 50.4364 9.0365 32.4766 28.4367 18.5468 31.4869 4.4570 54.7671 9.4572 19.7373 33.7174 59.8475 22.7376 15.5477 22.1478 21.8179 53.1080 38.7281 36.5882 39.2683 53.7184 40.9585 24.8286 45.0587 25.3088 58.3189 25.3090 9.4891 15.9392 25.0093 29.2794 26.2995 38.2696 31.3097 37.9798 38.5299 11.53

100 3.64

1 373 452 1146 55


Page 3-94 (Rev. 10/2004)

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/24/2004 POSTSTRATIFIED VARIABLE APPRAISAL Time: 9:51 AUDIT/REVIEW: Poststratification

DATA FILE USED: C:\Temp\POSTDATA.TXT

----------------------D I F F E R E N C E-----------------------Stratum 1 SAMPLE SIZE / UNIVERSE SIZE 45 373 MEAN 240.00 STANDARD DEVIATION 4.69 STANDARD ERROR (TOTAL) 347.13 POINT ESTIMATE 89,520

CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT 89,075 UPPER LIMIT 89,965 PRECISION AMOUNT 445 PRECISION PERCENT .50% Z-VALUE USED 1.281551565545

90% CONFIDENCE LEVEL LOWER LIMIT 88,949 UPPER LIMIT 90,091 PRECISION AMOUNT 571 PRECISION PERCENT .64% Z-VALUE USED 1.644853626951

95% CONFIDENCE LEVEL LOWER LIMIT 88,839 UPPER LIMIT 90,200 PRECISION AMOUNT 680 PRECISION PERCENT .76% Z-VALUE USED 1.959963984540

Stratum 2 SAMPLE SIZE / UNIVERSE SIZE 55 1,146 MEAN 30.00 STANDARD DEVIATION 14.09 STANDARD ERROR (TOTAL) 1,800.03 POINT ESTIMATE 34,379




(Rev. 10/2004) Page 3-95


OVERALL SAMPLE SIZE / UNIVERSE SIZE 100 1,519 POINT ESTIMATE 123,898 STANDARD ERROR 1,833




Discussion. Using the usual estimator for a simple random sample, the estimate of the

universe mean is

= [(45)(240) + (55)(30)]/100 = $124.50y

and the estimate of the universe total is

= (1519)(124.50) = $189,115.50$T

Since there is an unusually high number of inpatient bad debts (and low outpatient), a better

procedure would be to use the poststratified estimate of the universe total; namely

= (373)(240.00) + (1146)(30.00) = $123,900$Tpst


Page 3-96 (Rev. 10/2004)

The (more precise) computer-generated point estimate is $123,898 (highlighted).

Also, the point estimate for the inpatient stratum is = (373)(240.00) = $89,520 (highlighted); $T1

for the outpatient stratum, this estimate is = (1146)(30.00) = $34,380 (more precisely, the$T2

highlighted value of $34,379). Notice that = 89,520 + 34,379 = $123,899 (more precisely,$Tpst

$123,898).

The estimated variance of is $Tpst v T v T v Tpst( $ ) ( $ ) ( $ )= +1 2

= +⎡

⎣⎢⎤

⎦⎥+ +⎡

⎣⎢⎤

⎦⎥1419100

373 4 69484 1519100

1146 4 69484 1419100

1146 14 09098 1519100

373 14 0909822

2 22

2( )( . ) ( )( . ) ( )( . ) ( )( . )

= 120,499.88 + 3,240,111.44 = 3,360,611 and the estimated standard error is =3 360 611, ,

$1,833 (highlighted)

NOTE: The estimated standard error for stratum 1 is = 347.13 and the120 499 88, .estimated standard error for stratum 2 is = 1,800.03.3 240 11144, , .

The corresponding 95% confidence interval for the universe total is

123,898 ± 1.959963984540(1,833)

that is, $120,305 to $127,491.

Comments

1. Poststratification allows you to obtain a single simple random sample (easier than obtaining a

simple random sample from each stratum) and then stratify provided the strata sizes in the

universe (Ni) are known. The value of Ni is multiplied by the sample mean to estimateyi


(Rev. 10/2004) Page 3-97

the total for the i-th stratum. These estimates are then summed over all the strata to estimate

the universe total.

2. A minimum of 20 sampling units per stratum is required as well as 6 nonzero items per

stratum (OA Policy and Procedures). The total sample size should be at least 100.

3. With poststratification, the sample sizes (ni) are unknown in advance (random variables).

With stratified sampling, the sample sizes are fixed (nonrandom).

FORMULAS

1. Estimate of the universe total for the i-th stratum (Ti)

$T N yi i i=

where Ni = number of items (universe) in stratum i

= average of sample items in i-th stratumyi

L = number of strata

2. Estimate of universe total (T)

$ $T N y Tpst i i ii

L

i

L

= ===∑∑

11

3. Estimated variance of = where$Tpst v Tpst( $ ) = v Tii

L

( $ )=∑

1

v T N nn

N s Nn

N N si i i i i( $ ) ( )=−⎛

⎝⎜⎞⎠⎟

+ −22

2

where N = universe total = ENi and n is the total sample size.


Page 3-98 (Rev. 10/2004)

$ . ( $ )T v Ti i±1959963984540

$ . ( $ )T v Tpst pst±1959963984540

4. Estimated standard error of = $Ti v Ti( $ )

5. Approximate 95% confidence interval for stratum total (Ti):

6. Estimated standard error of is $Tpst v Tpst( $ )


NOTE: For a 90% confidence interval in equations 5 and 7, replace 1.959963984540 with1.644853626951 and for an 80% confidence interval replace 1.959963984540 with1.281551565545.

RAT-STATS Companion Manual Unknown Universe Size

(Rev. 10/2004) Page 3-99

Unknown Universe Size

This program calculates a confidence interval for a universe total when using variable sampling,

as does the Unrestricted Variable Appraisal program. When using the Unrestricted Variable

Appraisal program, one of the user queries is for the universe size (N) and this value must be

known. For situations where N is unknown, the Unknown Universe Size program can be used.

Use of this program requires that two random samples be used -- one to estimate the universe

size and the other to estimate one or more variable characteristics. Both samples must be

appraised using the Unrestricted Variable Appraisal program prior to running this module since

this program will ask for the mean and standard deviation of each sample.

The population of interest is a subset of some other universe. For example, the larger sampling

frame might consist of 575 file drawers containing a mixture of dental claims and the population

of interest consists of all claims related to a particular dental procedure (say, procedure ABC).

Suppose we sample 70 of the file drawers and count the number of claims related to procedure

ABC. The first step is to estimate the number of claims related to procedure ABC in all 575

drawers. The results on the next page were obtained and are stored in data file

DATAUNIV.TXT.

Unknown Universe Size RAT-STATS Companion Manual

Page 3-100 (Rev. 10/2004)

Number of Claims Related Sampled Drawer to Procedure ABC 1 9 2 12 3 9 Sample summary . . = 10.33x . . s = 2.75 . . 70 10 Total 723

Data file DATAUNIV.TXT <continued>

1 9 2 12 3 9 4 6 5 12 6 13 7 10 8 9 9 1010 611 1312 713 1214 715 1216 1217 1018 919 1020 1021 1322 1023 1024 1025 626 827 1428 2229 1030 831 832 833 1234 1035 8

36 937 1238 839 640 1441 942 1443 1444 645 1446 1247 948 1049 950 951 852 1453 954 1455 856 1257 1458 1059 1260 961 1262 1263 1464 1065 1066 667 968 869 1270 10

NOTE: This file contains 70lines.


(Rev. 10/2004) Page 3-101

The following computer output was obtained using the Unrestricted Variable Appraisal

program. In the Universe Size box, the size of the larger universe (575 file drawers here) was

used. With this procedure, you are able to see the estimated size of the universe of interest

( = 5,939) in this output.$N

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/24/2004 VARIABLE UNRESTRICTED APPRAISAL Time: 12:12 AUDIT/REVIEW: First Sample

DATA FILE USED: C:\Temp\DATAUNIV.TXT

SAMPLE SIZE VALUE OF SAMPLE NONZERO ITEMS 70 723.00 70

----------------------- E X A M I N E D ------------------------ MEAN / UNIVERSE 10.33 575 STANDARD DEVIATION 2.75 STANDARD ERROR .31 NOTE: Input to Unknown SKEWNESS 1.02 Universe Size program. KURTOSIS 5.95 POINT ESTIMATE 5,939 <--estimated universe size

CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT 5,710 UPPER LIMIT 6,168 PRECISION AMOUNT 229 PRECISION PERCENT 3.86% T-VALUE USED 1.293941609194

90% CONFIDENCE LEVEL LOWER LIMIT 5,644 UPPER LIMIT 6,234 PRECISION AMOUNT 295 PRECISION PERCENT 4.97% T-VALUE USED 1.667238548669

95% CONFIDENCE LEVEL LOWER LIMIT 5,586 UPPER LIMIT 6,292 PRECISION AMOUNT 353 PRECISION PERCENT 5.95% T-VALUE USED 1.994945415107


Page 3-102 (Rev. 10/2004)

The next step is to independently obtain a random sample from the population of interest to

appraise the variable(s) of interest. It was decided to sample 55 claims related to procedure ABC

and record the amount in error for each sampled claim. The results are stored in data file

DATAVAR.TXT.

Data file DATAVAR.TXT < - continued - >

1 15.05 2 14.05 3 18.55 4 13.62 5 18.78 6 6.97 7 17.17 8 23.75 9 19.3210 21.3211 25.1712 14.1213 15.8514 13.5015 14.6216 6.9717 22.6218 6.9719 15.6120 12.6921 21.4522 21.6223 17.9524 14.6525 6.9726 18.8927 15.3428 6.98

29 16.0730 20.1031 17.2932 21.3833 16.6034 6.9835 16.8536 19.2537 20.3038 29.5139 6.9540 19.0041 13.9342 12.4543 15.2344 15.0945 6.9846 20.4847 6.9848 16.4749 19.7550 6.9851 17.6552 15.2653 6.9854 14.1555 22.05

NOTE: This file contains 55lines.


(Rev. 10/2004) Page 3-103

This file is used as input to the Unrestricted Variable Appraisal program. The user can enter

any value in the Universe Size box since this value has no effect on the results produced by the

Unknown Universe Size program. The following computer output was produced.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/24/2004 VARIABLE UNRESTRICTED APPRAISAL Time: 13:12 AUDIT/REVIEW: Second Sample

DATA FILE USED: C:\Temp\DATAVAR.TXT

SAMPLE SIZE VALUE OF SAMPLE NONZERO ITEMS 55 860.76 55

Any value can be used for universe --------------------- D I F F E R E N C E ---------------------- MEAN / UNIVERSE 15.65 1,000 STANDARD DEVIATION 5.45 STANDARD ERROR .71 NOTE: Input to Unknown SKEWNESS -.14 Universe Size program. KURTOSIS 2.63 POINT ESTIMATE 15,650

CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT 14,723 UPPER LIMIT 16,577 PRECISION AMOUNT 927 PRECISION PERCENT 5.92% T-VALUE USED 1.297426488209




Page 3-104 (Rev. 10/2004)

Comment: This is the same example used in the RAT-STATS User’s Guide illustration. The

following computer output is produced by the Unknown Universe Size program:

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/24/2004 VARIABLE APPRAISAL WITH UNKNOWN UNIVERSE SIZE Time: 13:48 AUDIT/REVIEW: Variable Unknown Universe Size

= = = = = = = = = = = = = = = = I N P U T = = = = = = = = = = = = = = = = SAMPLE TO SAMPLE FOR ESTIMATE POPULATION VARIABLE ATTRIBUTE UNIVERSE 575 SAMPLE 70 55 MEAN 10.33 15.65 STANDARD DEVIATION 2.75 5.45

= = = = = = = = = = = = = E S T I M A T I O N = = = = = = = = = = = = = 80% CONFIDENCE 90% CONFIDENCE 95% CONFIDENCE

POINT ESTIMATE --> 92,957 92,957 92,957$T STANDARD ERROR 5,152 5,152 5,152 LOWER LIMIT 86,355 84,483 82,859 UPPER LIMIT 99,560 101,431 103,055 PRECISION AMOUNT 6,603 8,474 10,098 PRECISION PERCENT 7.10% 9.12% 10.86% Z-VALUE USED 1.281551565545 1.644853626951 1.959963984540

Discussion. The estimated total dollars in the universe is equal to

(estimated universe size)(mean of the variable sample)

= [(10.33)(575)](15.65) = $92,957. This is .$T

Using the formula section, the value of se1 is

( . )( ) .2 75 57570

1 70575

177118− =

and the value of se2 is

54555

1 555939

7315. .− =


(Rev. 10/2004) Page 3-105

The estimated variance of is$T

[(15.65)(177.118)]2 + [(5939)(.7315)]2 - [(177.118)(.7315)]2 = 26,540,250.

The estimated standard error of is = $5,152.$T 26 540 250, ,The PRECISION AMOUNT at the 90% confidence level is (5,152)(1.644853626951) = $8,474

(highlighted). This amount is 9.12% of the point estimate, since 8,474/92,957 = .0912.

The corresponding confidence interval is 92,957 ± 8,474; that is, $84,483 to $101,431.

FORMULAS

Given: Larger universe size: N1. Sample to estimate universe size: sample size is n1, mean is

, and standard deviation is s1.x1

Sample to estimate variable of interest: sample size is n2, mean is , and standardx2

deviation is s2.

1. Estimate of universe (of interest) size

$ •N N x= 1 1

2. Overall estimate for variable total (e.g., total error amount)

$ $ •T N x= 2


v T x se N se se se( $) [ • ] [ $ • ] [ • ]= + −2 12

22

1 22

where

and se s Nn

nN1

1 1

1

1

1

1= − se sn

nN2

2

2

21= −$


Page 3-106 (Rev. 10/2004)

$ . ( $)T v T±1959963984540



Reference:Estimated variance: Kendall's Advanced Theory of Statistics, Volume 1, 5th ed., Alan Stuartand J. Keith Ord, New York: Oxford University Press, 1987, page 343, exercise 10.23.

RAT-STATS Companion Manual Sample Size Determination

(Rev. 10/2004) Page 4-1

# Variable

• Unrestricted Using a Probe Sample

• Unrestricted Using Estimated Error Rate

• Stratified (Total Sample Size Known)

• Stratified (Total Sample Size Unknown)

SAMPLE SIZE DETERMINATION

A commonly encountered question in auditing is “How large a sample is necessary?” When

using an unrestricted (simple random) sample, this depends on the desired precision of the point

estimate. The programs in this section are listed below and are concerned with determining

sample sizes for various data types and sample strategies.

Unrestricted Variable Sample Size Determination RAT-STATS Companion Manual

Page 4-2 (Rev. 10/2004)

Variable Sample Size Determination

This RAT-STATS module can be used for two situations.

Situation 1: The program will help select the necessary sample size for an unrestricted or

stratified variable appraisal. The program output includes sample sizes for each stratum that will

provide precision percentages of 1%, 2%, 5%, 10%, 20% and “Other.” When selecting “Other,”

the user will be prompted to enter the desired precision percentage. The user may also select any

combination of the following confidence levels: 80%, 90%, 95%, and 99%.

Situation 2: The program also allows the user to determine the optimum distribution of a

sample among strata when the overall sample size has already been determined. It will allocate

the larger samples to those strata that are larger in size and/or contain a larger amount of

variation (are nonhomogeneous). Any combination of the confidence levels 80%, 90%, 95%,

and 99% can be selected.

Variable Sample Size Determination - Unrestricted Using a Probe Sample

This program allows the user to estimate sample sizes for specified precision percentages and

specified confidence levels. The user has the option of having the program read a probe sample

file to obtain an estimate of the universe mean and standard deviation or input these two

estimates directly without reading a probe sample file. The probe sample may be stored in a text

file, an Excel spreadsheet, or an Access table.

RAT-STATS Companion Manual Unrestricted Variable Sample Size Determination

(Rev. 10/2004) Page 4-3

Example 1. This example illustrates Situation 1. A probe sample of 25 examined values was

obtained. The audit objective was to determine the necessary sample sizes when estimating the

total examined amount for the universe of 100,000 items. The probe sample

(SAMPDATA.TXT) is shown below. The sample mean is $400 and the sample standard

deviation is $50.

321382453459343388313420407395441448447333357395477391356368376350461472447

The input screen and resulting text file output are shown on the following page.


Page 4-4 (Rev. 10/2004)

The following text file output is obtained using the previous screen. A sample size under 30 will

be flagged using “(*)” and the note immediately following the sample sizes will appear.


(Rev. 10/2004) Page 4-5

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 5/11/2004 Sample Size Determination Time: 21:52

Confidence Level 80% 90% 95% 99% 1% 256 421 597 1026 2% 64 106 150 259Precision 5% 10 (*) 17 (*) 24 (*) 41Level 10% 3 (*) 4 (*) 6 (*) 10 (*) 15% 1 (*) 2 (*) 3 (*) 5 (*) 25% --- 1 (*) 1 (*) 2 (*)

Estimated Mean: 400.00Estimated Std. Deviation: 50.00Universe Size: 100,000

NOTE (*): One or more sample sizes were under 30. Thegenerated sample sizes were the result ofmathematical formulas and did notincorporate management decisions concerningthe purpose of the sample or currentorganizational sampling policies. You mayneed to increase the sample sizes in orderto be in compliance with organizationalobjectives.

Explanation of Output

The output for each cell in the output table will consist of (1) the necessary sample size or (2) the

text “- - -”. The necessary sample size is the number of sample items necessary to obtain the

specified sample precision at the specified confidence level. For example, in this illustration, a

sample size of 106 is necessary to obtain a point estimate having a precision percentage of plus

or minus 2% using a 90% confidence level. If the calculated sample size is 0, a text value of “- -

-” will appear in this cell. This occurred in the lower left cell for the sample illustration.

The output also contains the estimated mean and standard deviation, along with the specified

universe size.


Page 4-6 (Rev. 10/2004)

FORMULAS

Let PREC = the precision percentage (e.g., 1 for 1%, 10 for 10%)

ZVAL = the value from the standard normal (Z) distribution having a right-tail area equalto (100 - Confidence Level)/2, where the right-tail area is expressed as aproportion between 0 and 1.

ZVAL is 1.281551565545(80%), 1.644853626951 (90%), 1.959963984540(95%), and 2.575829303549 (99%).

N = the universe size

Mean = estimated universe mean obtained from the probe sample or specified by the user

StdDev = estimated universe standard deviation obtained from the probe sample orspecified by the user

E = maximum error = (PREC/100) A Mean A N

For each selected value of PREC and ZVAL, the sample size is

n StdDev NE ZVAL N StdDev

=⋅

+ ⋅( )

( / ) ( )

2

2 2

The value of n is rounded up or down to the nearest integer.


(Rev. 10/2004) Page 4-7

Variable Sample Size Determination - UnrestrictedUsing Expected Error Rate

This procedure estimates the mean and standard deviation of the difference (error) amounts by

assuming (1) any item found to be in error is 100% in error and (2) the mean and standard

deviation of the nonzero error amounts is the same as the mean and standard deviation of the

reported (examined) amounts. The mean and standard deviation of the error amounts are

estimated by assuming the percentage of nonzero errors in the error population is equal to the

expected error rate (one of the input values) and the nonzero errors resemble the reported

amounts; that is, the mean and standard deviation of the nonzero errors are equal to the mean and

standard deviation of the reported amounts.

Comment. Even though these assumptions may not be entirely true, this procedure will often

give more reliable sample size estimates than those obtained using the Variable Unrestricted

(Using Reported Amounts) module since the expected number of zero values in the error

population is factored into the sample size calculation.

Example 2. This example illustrates another method of dealing with Situation 1. The estimated

error rate is 15% for a universe of 10,000 transactions. The total reported amount is $3,000,000

and the standard deviation of the reported amounts is $125. Consequently, the mean reported

amount is $300. Of interest is the required sample size necessary in order to obtain plus or

minus 15% using a 90% confidence level. The corresponding input screen follows where 25%

was specified for the “Other” precision level.


Page 4-8 (Rev. 10/2004)

The text file output shown on the next page is obtained using this input screen. A sample size

under 30 will be flagged using “(*)” in the program output and a note informing the user of this

fact will also appear in the program output immediately following the sample sizes.


(Rev. 10/2004) Page 4-9


Confidence Level 80% 90% 95% 99% 1% 9181 9486 9633 9784 2% 7370 8219 8676 9188Precision 5% 3095 4248 5119 6443Level 10% 1008 1559 2077 3117 15% 474 758 1044 1675 25% 176 287 403 675

Universe Size: 10,000

Anticipated Error Rate: 15%

Reported Amounts - - Total Amount: 3,000,000.00

Standard Deviation: 125.00

Difference Values - - Estimated Mean: 45.00

Estimated Standard Deviation: 117.55




specified sample precision at the specified confidence level. For example, in this illustration, a

sample size of 287 is necessary to obtain a point estimate having a precision percentage of plus

or minus 15% using a 90% confidence level. If the calculated sample size is 0, a text value of

“- - -” will appear in this cell.

The output also contains the estimated mean and standard deviation of the difference (error)

values. For this illustration, the estimated mean and standard deviation are $45.00 and $117.55,

respectively.


Page 4-10 (Rev. 10/2004)

FORMULAS

Let PREC = the precision percentage (e.g., 1 for 1%, 10 for 10%)

ZVAL = the value from the standard normal (Z) distribution having a right-tail area equalto (100 - Confidence Level)/2, where the right-tail area is expressed as aproportion between 0 and 1.

ZVAL is 1.281551565545(80%), 1.644853626951 (90%), 1.959963984540(95%), and 2.575829303549 (99%).

N = the universe size (input)

TR = the total reported amount (input)

:R = mean reported amount = TR / N

= standard deviation of reported amounts (input)σ R

= the estimated error rate (input)$p

= the estimated mean of the difference (error) values = $µD $p Rµ

= estimated standard deviation of the difference (error) values$σD

= $[ ( $) ]p pR Rσ µ2 21+ −

E = maximum error = (PREC/100) A A N$µD

For each selected value of PREC and ZVAL, the sample size is

nN

E ZVAL ND

D

=⋅+ ⋅

( $ )( / ) ( $ )

σσ

2

2 2

The value of n is rounded up or down to the nearest integer.

RAT-STATS Companion Manual Stratified Variable Sample Size Determination

(Rev. 10/2004) Page 4-11

Variable Sample Size Determination - Stratified

Stratified Sample Sizes - Total Sample Size is Unknown

Example 3. This example illustrates Situation 1. Two strata have been defined: The high-

income stratum (N1 = 100,000 items) and the low-income stratum (N2 = 500,000 items). Of

interest is the total audit (claimed) amount for the universe. For the high-income stratum, the

estimated mean of the audited amounts is $10,000 and the estimated standard deviation is

$5,000. These values for the low-income stratum are $5,000 (mean) and $4,000 (standard

deviation). At a confidence level of 95%, what sample size is required to obtain a precision

percentage of ± 10%?

Solution. The following input screen was used for this example.

Stratified Variable Sample Size Determination RAT-STATS Companion Manual

Page 4-12 (Rev. 10/2004)

The following output is obtained using the previous screen. If one or more of the sample sizes

are under 30, the note immediately following the total sample sizes will appear.


THE ESTIMATES ARE BASED ON THE FOLLOWING ENTRIES:

NBR DESCRIPTION -- MEAN -- -- STD.DEV. -- -- UNIVERSE -- -- RATIO -- 1 High Income 10,000.00 5,000.00 100,000 20.00% 2 Low Income 5,000.00 4,000.00 500,000 80.00%

- TOTALS - 5,833.33 4,579.54 600,000

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

Sample Sizes for Stratum 1: High Income

Confidence Level 80% 90% 95% 99% 1% 1653 2699 3795 6406 2% 418 687 972 1669Precision 5% 67 111 157 271Level 10% 17 (*) 28 (*) 40 68 15% 8 (*) 13 (*) 18 (*) 31 25% 3 (*) 5 (*) 7 (*) 11 (*)

Sample Sizes for Stratum 2: Low Income

Confidence Level 80% 90% 95% 99% 1% 6611 10793 15180 25624 2% 1671 2745 3888 6676Precision 5% 268 442 627 1081Level 10% 68 111 157 271 15% 30 50 70 121 25% 11 (*) 18 (*) 26 (*) 44

Total Sample Sizes

Confidence Level 80% 90% 95% 99% 1% 8264 13492 18975 32030 2% 2089 3432 4860 8345Precision 5% 335 553 784 1352Level 10% 85 139 197 339 15% 38 63 88 152 25% 14 (*) 23 (*) 33 55


(Rev. 10/2004) Page 4-13

NOTE (*): One or more sample sizes were under 30. The generatedsample sizes were the result of mathematical formulasand did not incorporate management decisionsconcerning the purpose of the sample or currentorganizational sampling policies. You may need toincrease the sample sizes in order to be in compliancewith organizational objectives.

If any of the calculated samples sizes exceeds the corresponding universe size, the program will

conclude with the following reminder:

NOTE (#): The formulas calculated a sample size greater than theuniverse size. The program reduced the calculatedsample size to the universe size. The additionalsampling units were then distributed among theremaining strata based on optimal allocation formulas.

Discussion. For 10% precision and 95% confidence, the total sample size required is n = 197

with n1 = 40 items to be obtained from the high-income stratum and n2 = 157 from the low-

income stratum. Consequently, a 95% confidence interval based on these sample sizes should

result in a precision percentage of ±10%. This assumes that the resulting sample means and

standard deviations are the same as the values used as input to this program.

To demonstrate this, a data set was constructed that contained 40 items from stratum 1 with a

sample mean and standard deviation of $10,000 and $5,000, respectively, and 157 items from

stratum 2 with a sample mean and standard deviation of $5,000 and $4,000, respectively. When

this data set (named STRATA.TXT) was used as input to the STRATIFIED VARIABLE

APPRAISAL module, the computer output on the next page was obtained. In the final portion of

the output, notice that the resulting point estimate for the universe total is 3,500,000,000. At the

95% confidence level, the precision amount is 349,043,863 and is in fact (approximately) 10% of

the point estimate.


Page 4-14 (Rev. 10/2004)

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/23/2004 STRATIFIED VARIABLE APPRAISAL Time: 10:36 AUDIT/REVIEW: Two Strata Example

DATA FILE USED: C:\Temp\STRATA.TXT


TOTALS 197 1,185,000.00 197

--------------------- D I F F E R E N C E ----------------------Stratum 1 MEAN / UNIVERSE 10,000.00 100,000 STANDARD DEVIATION 5,000.04 µ Approx. 5,000 STANDARD ERROR 790.42 SKEWNESS -.22 KURTOSIS 2.30 STANDARD ERROR (MEAN) 790.42 STANDARD ERROR (TOTAL) 79,041,756 POINT ESTIMATE 1,000,000,000

CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT 896,958,117 UPPER LIMIT 1,103,041,883 PRECISION AMOUNT 103,041,883 PRECISION PERCENT 10.30% T-VALUE USED 1.303638588621

90% CONFIDENCE LEVEL LOWER LIMIT 866,824,512 UPPER LIMIT 1,133,175,488 PRECISION AMOUNT 133,175,488 PRECISION PERCENT 13.32% T-VALUE USED 1.684875121711

95% CONFIDENCE LEVEL LOWER LIMIT 840,122,958 UPPER LIMIT 1,159,877,042 PRECISION AMOUNT 159,877,042 PRECISION PERCENT 15.99% T-VALUE USED 2.022690920037

Stratum 2 MEAN / UNIVERSE 5,000.00 500,000 STANDARD DEVIATION 3,999.81 µ Approx. 4,000 SKEWNESS .87 KURTOSIS 3.30 STANDARD ERROR (MEAN) 319.17 STANDARD ERROR (TOTAL) 159,584,887 POINT ESTIMATE 2,500,000,000


(Rev. 10/2004) Page 4-15

CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT 2,294,613,944 UPPER LIMIT 2,705,386,056 PRECISION AMOUNT 205,386,056 PRECISION PERCENT 8.22% T-VALUE USED 1.287001917850

90% CONFIDENCE LEVEL LOWER LIMIT 2,235,938,079 UPPER LIMIT 2,764,061,921 PRECISION AMOUNT 264,061,921 PRECISION PERCENT 10.56% T-VALUE USED 1.654679995672

95% CONFIDENCE LEVEL LOWER LIMIT 2,184,773,966 UPPER LIMIT 2,815,226,034 PRECISION AMOUNT 315,226,034 PRECISION PERCENT 12.61% T-VALUE USED 1.975287507703

OVERALL POINT ESTIMATE / UNIVERSE 3,500,000,000 600,000 STANDARD ERROR 178,086,876

CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT 3,271,772,485 UPPER LIMIT 3,728,227,515 PRECISION AMOUNT 228,227,515 PRECISION PERCENT 6.52% Z-VALUE USED 1.281551565545

90% CONFIDENCE LEVEL LOWER LIMIT 3,207,073,156 UPPER LIMIT 3,792,926,844 PRECISION AMOUNT 292,926,844 PRECISION PERCENT 8.37% Z-VALUE USED 1.644853626951

95% CONFIDENCE LEVEL LOWER LIMIT 3,150,956,137 UPPER LIMIT 3,849,043,863 PRECISION AMOUNT 349,043,863 PRECISION PERCENT 9.97% µ Z-VALUE USED 1.959963984540

Comments

(1) When the sample of size n = 197 is obtained, the values of the sample mean and standard

deviation will likely not be exactly those specified in the input to this program.


Page 4-16 (Rev. 10/2004)

Consequently, the best the user can hope for is that the resulting precision percentage will

be approximately 10%.

(2) For the preceding example, the specified precision was 10% of the point estimate. The

point estimate for the universe total was 3,500,000,000. In the formula section, E is the

desired precision amount expressed as a percentage of the point estimate for the universe

total. Here this would be E = 350,000,000.

(3) For situations in which you do not have an estimate of the universe standard deviation (F)

from previous audit results, a rough approximation for F can be obtained for each stratum

by estimating (1) the largest value (L) that you expect to see in the sample for this stratum

and (2) the smallest value (S) that you expect to see in this stratum. Then, the approximate

value of F for this stratum is . In the previous example, if the largest audit$σ =−L S4

amount that you expect to see in the LOW INCOME stratum is L = $15,000 and the

smallest value is S = $1,000, then the estimated standard deviation is = (15,000 -$σ

1,000)/4 = $3,500.

Stratified Sample Sizes - Total Sample Size is Known

Example 4. This is an illustration of situation 2. The situation is the same as that described in

Example 3, which used two strata -- the high-income stratum and the low-income stratum. The

total sample size is set at 500. The input screen on the following page was used for this

example. Notice that the user is unable to set the precision percentages for this situation.


(Rev. 10/2004) Page 4-17

The following estimates were used as input to the program:

Estimated Estimated EstimatedStratum Mean Standard Deviation Universe SizeHigh Income 10,000 5,000 100,000Low Income 5,000 4,000 500,000

The program output on the next page is obtained. Notice that the resulting strata ratios (i.e., 20%

and 80%) are identical to those obtained in Example 3.


Page 4-18 (Rev. 10/2004)



NBR DESCRIPTION -- MEAN -- -- STD.DEV. -- -- UNIVERSE -- 1 High Income 10,000.00 5,000.00 100,000 2 Low Income 5,000.00 4,000.00 500,000

- TOTALS - 5,833.33 4,579.54 600,000= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

Precision Values: NOTE: See the Discussion section.

Confidence Level 80% 90% 95% 99% 4.09% 5.25% 6.26% 8.22%

The following sample sizes are based on a total sample size of 500.

Stratum 1: High Income Sample Size Ratio 100 20.00%

Stratum 2: Low Income Sample Size Ratio

400 80.00%

Discussion. The two sample sizes are n1 = 100 and n2 = 400, which total n = 500. For this

example, is (100,000)(5,000) + (500,000)(4,000) = 2,500,000,000. Call this SUM. ∑ Ni i$σ

The ratio value for stratum 1 is (100,000)(5,000) divided by SUM; that is .2. So, 20% of the

sample size is allocated to stratum 1; that is, n1 is (500)(.2) = 100. Similarly, the ratio for stratum

2 is .8 and n2 is (500)(.8) = 400. NOTE: This same discussion applies to Example 3.

What is the precision amount for this sampling design? This will be the value obtained by the

Stratified Variable Appraisal program using these sample sizes and estimated standard

deviations. This formula (borrowed from the Stratified Variable Appraisal formula section) is

contained in the formula section to follow. For this example, the precision amount will be


(Rev. 10/2004) Page 4-19

195996 100 000 100 000 100100 000

5 000100

500 000 500 000 400500 000

4 000400

22

22

. , ,,

, , ,,

,−⎛⎝⎜

⎞⎠⎟

+−⎛

⎝⎜⎞⎠⎟

= 219,038,136.

The estimated universe total is

= E(stratum mean)(stratum size) = (10,000)(100,000) + (5,000)(500,000) = 3,500,000,000.$T

The resulting precision percentage is

100 A (219,038,136 / 3,500,000,000) = 6.26%.

This value is called PERC in the formula section to follow and matches with the highlighted

value in the computer output.

FORMULAS

Total Sample Size (n) is Known

Notation

L = Number of strata

Ni = the universe size for the i-th stratum

(StdDev)i = estimated universe standard deviation for the i-th stratum

SUM = N StdDevi ii

L

⋅=∑ ( )

1

(Ratio)i = [Ni A (StdDev)i] / SUM

The resulting sample size allocated to the i-th stratum is ni = n A (Ratio)i .


Page 4-20 (Rev. 10/2004)

Total Sample Size (n) is Unknown

Notation

L = Number of strata

Ni = the universe size for the i-th stratum

N = the total universe size = Nii

L

=∑

1(Mean)i = estimated universe mean for the i-th stratum

UnivTotal = estimated universe total = N Meanii

L

i=∑ ⋅

1

( )

(StdDev)i = estimated universe standard deviation for the i-th stratum

SUM1 = N StdDevi ii

L

⋅=∑ ( )

1

SUM2 = N StdDevi ii

L

⋅=∑ ( )2

1

(Ratio)i = [Ni A (StdDev)i] / SUM1

PREC = the precision percentage (e.g., 1 for 1%, 10 for 10%)

ZVAL = the value from the standard normal (Z) distribution having a right-tail area equalto (100 - Confidence Level)/2, where the right-tail area is expressed as aproportion between zero and one.

ZVAL is 1.281551565545 (80%), 1.644853626951 (90%),1.959963984540 (95%), and 2.575829303549 (99%).

E = the precision amount = (PREC/100) A (UnivTotal)


(Rev. 10/2004) Page 4-21

For each selected value of PREC and ZVAL,

(1) the total sample size is

nSUM

E ZVAL SUM=

+( )

( / )1

2

2

2

(2) the sample size allocated to the i-th stratum is

ni = n A (Ratio)i

Comments

1. In the preceding calculation, the value of n is treated as a floating point number (e.g.,n = 487.263) and the strata sample sizes (ni) are calculated using this value. The ni valuesare then rounded up to the nearest integer. After all strata sample sizes have beendetermined, n is reset to the sum of the ni.

2. If the computed sample size for stratum i (ni) is larger than the universe size Ni, then ni isset equal to Ni. The remaining sample sizes are then obtained by applying the aboveformula and (1) omitting the i-th stratum in the denominator and (2) replacing n with n -Ni (the total sample size for the remaining L-1 strata).

The precision percentage at the 95% confidence level is ± PERC, where

PERCT

NN n

NStdDev

nii i

i

i

ii

L

=−⎛

⎝⎜

⎞⎠⎟

=∑1959963984540 2

2

1

.$

( )

and where is the estimated total for the universe. The value of is obtained by multiplying$T $T

Ni by the estimated mean for stratum i and summing over the L strata; that is, $T = Ni ii

L$µ

=∑

1

NOTE: Replace 1.959963984540 with 1.281551565545 for an 80% interval,1.644853626951 for a 90% interval, and 2.575829303549 for a 99% interval.

Attribute Sample Size Determination RAT-STATS Companion Manual

Page 4-22 (Rev. 10/2004)

Attribute Sample Size Determination

This program determines the sample size for an attribute simple random sample. The sample

size is determined for specified degrees of precision (using the desired width of the confidence

intervals) and for various levels of confidence. The resulting sample size is the smallest sample

size capable of meeting the specified precision requirement at each of the specified confidence

levels. The user may select any combination of the following confidence levels: 80%, 90%,

95%, and 99%.

Confidence intervals for attribute sampling are exact and are based on the hypergeometric

distribution. As a result, such confidence intervals are usually not symmetric about the point

estimate. For example, the point estimate might be 3% and the corresponding 95% confidence

interval might be 2% to 6%. For this illustration, the width of the confidence interval is 4% and

the confidence level is 95%. Consequently, attribute confidence intervals differ from the usual

interval obtained by deriving the point estimate plus or minus the estimated precision, where the

estimated precision is half the width of the resulting confidence interval. Because of this, the

“desired precision” for the attribute sampling procedure must be specified as the desired width

(rather than the half-width) of the confidence interval.

An approximate confidence interval for a universe proportion (discussed in many introductory

statistics textbooks) is based on the normal approximation. This particular interval follows the

“usual” procedure where the confidence interval is equal to (point estimate) ± (estimated

precision); that is, this interval is symmetric about the point estimate. However, this confidence

interval is approximate and is unreliable whenever the estimated proportion is very small or very

RAT-STATS Companion Manual Attribute Sample Size Determination

(Rev. 10/2004) Page 4-23

large, unless the sample size is extremely large. The confidence interval using the RAT-STATS

attribute sample size module discussed here is always exact.

The input screen includes (1) the size of the universe and (2) the anticipated rate of occurrence in

the universe. This rate of occurrence is generally estimated from past experience, either from

similar systems or a past review of this universe. If no information concerning the rate of

occurrence is available, the most conservative procedure is to specify 50% for this value. If the

actual rate of occurrence differs from the user-specified rate of occurrence, this in no way affects

the sample's validity but the resulting precision (confidence interval width) may not meet the

desired precision requirement.

Example 5. An audit is to be carried out using a universe of N = 10,000 documents to determine

what proportion (p) of the documents do not have the proper approval signature. A confidence

level of 95% will be used. It is estimated that 20% of the documents will not have the proper

signature. Consequently, the estimate of p is = .20.$p

NOTE: This may be a rough guess if little information regarding this estimate is availablefrom previous audit experience. If the user has no idea as to the value of p, = .5$pshould be used. This will produce the largest possible sample size (for fixed valuesof N and precision range) but the user will be guaranteed that the resulting confidenceinterval will meet the desired precision range.

Suppose that the desired precision range is 6%. This is equal to the desired value of (upper

confidence limit - lower confidence limit). If the confidence limits were symmetric about the

point estimate, the user would have specified the precision as ± 3% for this situation, where 3%

is half the width of the resulting confidence interval. Since the exact procedure used in this


Page 4-24 (Rev. 10/2004)

program usually does not produce an interval symmetric about the point estimate, the user must

specify the desired total width of the confidence interval. The following input screen is used for

this example:

The resulting computer output (saved to a text file) is shown on the next page.


(Rev. 10/2004) Page 4-25


Confidence Level 80% 90% 95% 99%Sample Size 314 488 666 1,079

Anticipated Rate of Occurrence: 20%

Desired Precision Range: 6%





specified sample precision at each confidence level. For example, in this illustration a sample

size of 488 is necessary to obtain a confidence interval having a width of 6% using a 90%

confidence level. If the calculated sample size is zero, a text value of “- - -” will appear in this

cell.

Discussion. The necessary sample size (highlighted) is n = 666. As a result, after the sample

of 666 is obtained, if the resulting point estimate is close to = .20, then the resulting 95%$p

confidence interval for p should have a width approximately equal to .06 (such as .1710 to .2310,

with a width of .2310 - .1710 = .06). If the resulting sample produced 133 documents not

containing the proper signature, then the rate of occurrence in this sample would be 133/666; that

is, 20%.

The resulting confidence interval will have a width equal to .06 (i.e., 6%). This can be seen in

the computer output below, obtained using the Unrestricted Attribute Appraisal module. The


Page 4-26 (Rev. 10/2004)

width of this 95% confidence interval is 23.10% -17.10% = 6% (the desired precision range).

Notice that = .20 (i.e., 20%) is inside this interval (it always is), but it is not in the center.$p

Department of Health and Human Services OIG - Office of Audit ServicesDate: 10/19/2004 Single Stage Attribute Appraisal Time: 12:24 AUDIT/REVIEW: Example

UNIVERSE SIZE 10,000 SAMPLE SIZE 666 CHARACTERISTIC(S) OF INTEREST QUANTITY IDENTIFIED IN SAMPLE 133 PROJECTED QUANTITY IN UNIVERSE 1,997 PERCENT 19.970% STANDARD ERROR PROJECTED QUANTITY 150 PERCENT 1.498%

CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT - QUANTITY 1,805 PERCENT 18.050% UPPER LIMIT - QUANTITY 2,202 PERCENT 22.020%

90% CONFIDENCE LEVEL LOWER LIMIT - QUANTITY 1,754 PERCENT 17.540% UPPER LIMIT - QUANTITY 2,259 PERCENT 22.590%



(Rev. 10/2004) Page 4-27

Example 6. Repeat Example 5 where no information is available regarding the proportion of

documents not containing the proper signature.

Solution. Here, the user should enter 50% ( = .5) in the Anticipated Error Rate box. The$p

resulting computer output is shown below. The necessary sample size (highlighted) is now

n = 991, approximately 50% larger than the previous sample size of 666.


Confidence Level 80% 90% 95% 99%

Sample Size 466 725 991 1,580




Discussion. Example 6 illustrates how using = .5 produces a very large value of n. The$p

user should be encouraged to use even a rough guess for the value of . Using = .5 is a$p $p

very conservative procedure because with a sample of size n = 991, quite likely the resulting

confidence interval will have a width considerably less than the desired precision range of 6%.

To illustrate, the computer output below was obtained when the sample of 991 documents

produced 248 not containing the proper signature. Here, = 248/991 = .250 and the confidence$p

interval width (using the highlighted values in the following computer output) is 5.21%. This

value is less than 6%, but the user did have the guarantee that this value would be no more

than 6%.


Page 4-28 (Rev. 10/2004)

Department of Health and Human Services OIG - Office of Audit ServicesDate: 10/19/2004 Single Stage Attribute Appraisal Time: 12:43 AUDIT/REVIEW: Example


CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT - QUANTITY 2,334 PERCENT 23.340% UPPER LIMIT - QUANTITY 2,678 PERCENT 26.780%




(Rev. 10/2004) Page 4-29

FORMULAS

In the discussion to follow, a sample item having the attribute of interest will be referred to as an

item “in error.” Consequently, the universe proportion, p, will be the “error rate.”

Consider the case where the specified confidence level is 95%. The upper limit of the 95%

confidence interval for the universe total is, say, k2, where k2 is the largest value of k for which

ki

N kn iNn

i

x

⎛⎝⎜⎞⎠⎟

−−

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

>=∑

0

025.

where N = universe size

n = sample size

k = total number of universe items in error

x = number of sample items in error

.025 = [1 - (confidence level)]/2

NOTE: Here, the “confidence level” is expressed as .95.

The lower limit of the 95% confidence interval is, say, k1, where k1 is the smallest value of k for

whichki

N kn iNn

i x

n⎛⎝⎜

⎞⎠⎟

−−

⎛⎝⎜

⎞⎠⎟

⎛⎝⎜

⎞⎠⎟

>=∑ .025

The resulting 95% confidence interval for the total number of universe items in error is k1 to k2.


Page 4-30 (Rev. 10/2004)

The procedure used to derive this confidence interval can be found in John P. Buonaccorsi

(1987), “A Note on Confidence Intervals for Proportions in Finite Populations,” The American

Statistician, Vol. 41, No. 3, pp. 215-218.

Suppose that the universe size is N = 10,000, the anticipated rate of occurrence (i.e., error rate) is

20%, and the desired precision range is 6%. Since (10,000)(.06) is 600, we know that k2 = k1 +

600; that is, the upper confidence limit must be 600 more than the lower limit. The anticipated

rate of occurrence is used to specify the number of sample items that contain the characteristic of

interest. Here, it would be 20% of n, where n is the sample size determined by this program.

For example, suppose that n = 300 and (300)(.20) = 60 (call this x). If the values, N = 10,000,

n = 300, and x = 60 are used as input to the Unrestricted Attribute Appraisal program, the

resulting 95% confidence interval for the universe proportion (p) has a lower limit of .1569 [i.e.,

k1 = (10,000)(.1569) = 1,569] and an upper limit of .2490 [i.e., k2 = (10,000)(.2490) = 2,490].

But this is not a satisfactory value of n since k2 - k1 = 2,490 - 1,569 = 921, which must equal 600

according to the previous discussion.

Summary of program procedure. For a specified confidence level of 95%, this program

searches for the value of n that produces a confidence interval (k1 to k2) such that k1 and k2

satisfy the preceding two inequalities and k2 - k1 = 600, where, in general, 600 is equal to N A

(desired precision range). For the preceding example, if n = 666, then (666)(.20) . 133. If the

values N = 10,000, n = 666, and x = 133 are used as input to the Unrestricted Attribute

Appraisal module, the resulting 95% confidence interval for the universe proportion (p) has a

lower limit of .1710 [i.e., k1 = (10,000)(.1710) = 1,710] and an upper limit of .2310 [i.e.,


(Rev. 10/2004) Page 4-31

k2 = (10,000)(.2310) = 2,310]. This is satisfactory, since k2 - k1 = 600 and the difference of the

two proportions is .06 (i.e., 6%).

Version 2

Page TOC-2 (Rev. 11/2005)

TABLE OF CONTENTS

PageGETTING STARTED

IntroductionInstallation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iFrequently Asked Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii

Windows Version of RAT-STATSOpening Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1Random Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2Attribute Appraisals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3Variable Appraisals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4Sample Size Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5

RANDOM NUMBERS

OverviewSingle Stage Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2Sets of Two Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2Sets of Three Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2Sets of Four Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2Frames - Single Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2Frames - Sets of Two . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3RHC Sample Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3

Single Stage NumbersPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-6Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-6 Output to a Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-6 Output to an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-8 Output to an Excel Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9 Output to a Flat File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-10 Output to a Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-12 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-13

(Rev. 11/2005) Page TOC- 3

Sets of Two NumbersPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-14Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-14Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-16Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-16 Output to a Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-17 Output to an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-18 Output to an Excel Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-19 Output to a Flat File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-21 Output to a Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-22 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-24

Sets of Three NumbersPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-25Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-25Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-27Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-27 Output to a Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-28 Output to an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-29 Output to an Excel Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-30 Output to a Flat File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-31 Output to a Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-32 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-34

Sets of Four NumbersPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-35Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-35Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-37Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-37 Output to a Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-38 Output to an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-39 Output to an Excel Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-40 Output to a Flat File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-41 Output to a Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-42 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-44

Frames - Single StagePurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-45Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-45Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-47Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-47 Output to a Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-48 Output to an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-49


Output to an Excel Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-50 Output to a Flat File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-51 Output to a Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-52 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-54

Frames - Sets of Two NumbersPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-55Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-55Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-57Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-58 Output to a Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-58 Output to an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-59 Output to an Excel Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-61 Output to a Flat File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-62 Output to a Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-63 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-65

RHC Sample SelectionPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-66Input Screen for Selecting Primary/Secondary Units . . . . . . . . . . . . . . . . . . . 2-67Selecting the Number of Primary/Secondary Units to be Sampled . . . . . . . . . 2-67Seed Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-68Input File Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-68Format of Input File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-69 Input from a Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-70 Input from an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-71 Input from an Excel Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-73Output Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-73Format for Complete Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-74Format for Summary Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-74 Text Summary File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-75 Access Summary File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-75 Excel Summary File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-75Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-76 Complete Output to Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-77 Summary Output to Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-79 Summary Output to Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-80 Summary Output to Excel Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-80 Summary Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-81

(Rev. 11/2005) Page TOC- 5

ATTRIBUTE APPRAISALS

OverviewUnrestricted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2Stratified . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2Two-Stage Unrestricted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2Three-Stage Unrestricted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2RHC Two-Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2RHC Three-Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3Stratified Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3Stratified Multistage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3

UnrestrictedPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-5Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-6 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-7 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-10

StratifiedPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-12Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-12Entering the Sample Results from the Screen . . . . . . . . . . . . . . . . . . . . . . . . . 3-14Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19

Two-Stage UnrestrictedPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-20Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-20Entering the Sample Results from the Screen . . . . . . . . . . . . . . . . . . . . . . . . . 3-24Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-25Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-25 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-27 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-28

Three-Stage UnrestrictedPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-29Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-29Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-34Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-34


Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-36 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-38

RHC Two StagePurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-39Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-39Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-43Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-43 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-47 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-49

RHC Three StagePurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-50Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-50Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-56Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-57 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-60 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-63

Stratified ClusterPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-64Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-64Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-69Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-69 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-71 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-73

Stratified MultistagePurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-74Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-74Entering the Sample Results from the Screen . . . . . . . . . . . . . . . . . . . . . . . . . 3-77Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-77Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-78 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-79 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-80

VARIABLE APPRAISALS

OverviewUnrestricted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2Stratified . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2Two-Stage Unrestricted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2

(Rev. 11/2005) Page TOC- 7

Three-Stage Unrestricted . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2RHC Two-Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2RHC Three-Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3Stratified Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3Stratified Multistage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3Poststratification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3Unknown Universe Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3

UnrestrictedPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-5Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-5Format of Input File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-7 Input from a Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-8 Input from an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-9 Input from an Excel Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-12Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-13Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-14 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-15 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-18

StratifiedPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-19Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-19Format of Input File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-21 Input from a Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-22 Input from an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-25 Input from an Excel Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-28Complete or Summary Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-29Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-30Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-31 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-32 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-35

Two-Stage UnrestrictedPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-37Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-37Format of Input File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-39 Input from Text Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-40 Input from an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-43 Input from Excel Spreadsheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-46Specifying the Number of Secondary Units . . . . . . . . . . . . . . . . . . . . . . . . . . 4-48Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-49


Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-50 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-51 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-52

Three-Stage UnrestrictedPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-55Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-55Format of Input File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-57Format of Primary Unit Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-58Format of Secondary Unit Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-58Format of Sample Data Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-59 Input from Text Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-60 Input from an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-63 Input from Excel Spreadsheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-66Complete or Summary Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-69Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-69Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-69 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-71 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-73

RHC Two StagePurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-76Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-76Format of Input File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-78Format of Primary Unit Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-79 Input from Text Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-80 Input from an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-83 Input from Excel Spreadsheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-86Summary Screens . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-89Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-90Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-90 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-93 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-96

RHC Three StagePurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-98Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-98Format of Input File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-100Format of P.U. / S.U. Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-101 Input from Text Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-103 Input from an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-106 Input from Excel Spreadsheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-110

(Rev. 11/2005) Page TOC- 9

Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-113Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-113 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-116 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-119

Stratified ClusterPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-122Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-122Format of Input File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-124 Input from a Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-125 Input from an Access Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-126 Input from an Excel Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-129Complete or Summary Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-130Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-130Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-131 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-133 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-134

Stratified MultistagePurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-136Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-136Input of Data Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-138Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-139Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-139 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-141 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-142

PoststratificationPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-143Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-143Format of Input File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-145 Input from Text Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-146 Input from an Access Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-149 Input from Excel Spreadsheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-151Complete or Summary Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-153Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-154Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-155 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-156 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-158

Unknown Universe SizePurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-161

Page TOC-10 (Rev. 11/2005)

Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-161Specify Input Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-162 Sample Used to Estimate the Universe Size . . . . . . . . . . . . . . . . . . . . . . . . 4-163 Sample Used for Variable Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-163Output Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-164Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-164 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-166 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-166


OverviewVariable Sample Size Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-1Attribute Sample Size Determination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-1

Variable Sample Size Determination - Unrestricted Using Reported AmountsPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2Probe Sample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-3 Probe Sample in a Text File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-3 Probe Sample in an Excel Spreadsheet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-4 Probe Sample in an Access Database Table . . . . . . . . . . . . . . . . . . . . . . . . . . 5-6 No Probe Sample Used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-8Working with the Full Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-9 Confidence Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-9 Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-9Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-10Explanation of Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-10 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-10 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-11

Variable Sample Size Determination - Unrestricted Using Estimated Error RatePurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-13Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-13Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-14Working with the Full Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-14 Confidence Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-14 Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-14Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-15Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-16Explanation of Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-16 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-16 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-18

(Rev. 11/2005) Page TOC- 11

Variable Sample Size Determination - StratifiedPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-19Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-19Sample Size is Known / Unknown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-20Working with the Full Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-20 Confidence Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-20 Precision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-20Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-20Explanation of Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-21Program Output - Total Sample Size is Unknown . . . . . . . . . . . . . . . . . . . . . . 5-22 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-22 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-24Program Output - Total Sample Size is Known . . . . . . . . . . . . . . . . . . . . . . . . 5-26 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-26 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-27

Attribute Sample Size DeterminationPurpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-29Input Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-29Input Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-30 Confidence Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-30 Anticipated Rate of Occurrence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-30 Universe Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-31 Desired Precision Range . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-31Program Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-31Explanation of Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-31 Output to a Text File or Printer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-32 Output to Screen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-33

Page TOC-12 (Rev. 11/2005)

APPENDICES

Data Limitations for RAT-STATS 2007 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1Random Number Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1Attribute Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1Variable Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-2Sample Size Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-3

Numerical Accuracy in RAT-STATS 2007 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1

Troubleshooting for RAT-STATS 2007 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1

RAT-STATS User’s Guide INTRODUCTION

(Rev. 11/2005) Page i

DEPARTMENT OF HEALTH & HUMAN SERVICESOIG - OFFICE OF AUDIT SERVICES

RAT-STATS 2007VERSION 2

INTRODUCTION

RAT-STATS is a package of statistical software tools to assist the user in performing randomsamples and evaluating the results. The package is designed to run on personal computers usingMicrosoft’s Windows (Windows 95 and later versions).

We have attempted to make the software as “user friendly” as possible, keeping in mind the useris working with the technical terms of statistics. We have also attempted to make the softwareflexible in terms of entering data and allowing output to a variety of devices.

Installation

The executable version of this program is called RATSTATS2007V2.exe. This program can beinstalled by double clicking on SetupRS2007V2.exe. After installing RATSTATS2007V2.exe, the program can be executed by clicking on this file. An easier procedure would be to install ashortcut to RATSTATS2007V2.exe on the windows desktop.

The default installation location is C:\Program Files\RATSTATS2007. During the installationprocedure, the user has the option of selecting a different location. After the installation iscomplete, the folder containing RATSTATS2007V2.exe will also contain User Guide andCompanion Manual files in pdf format.

This guide contains explanations and examples for using each of the programs. To understandhow to use any particular option, look up the program in the table of contents and proceed to theappropriate pages for an overview, explanations, and examples.

INTRODUCTION RAT-STATS User’s Guide

Page ii (Rev. 11/2005)

Frequently Asked Questions

ARE THE RANDOM NUMBERS REALLY RANDOM?

The random number generator used throughout this package is based on an article entitled“Building a Random Number Generator” that appeared in the March 1987 issue of Bytemagazine (pages 127 and 128). The random number software in this package was tested using13 certification programs from the National Bureau of Standards to test for various aspects ofrandomness. The software passed all 13 of the tests.

WHAT IS THE COMPUTER LANGUAGE OF RAT-STATS?

The software was written using Microsoft Visual Basic (Version 6).

WHO IS RESPONSIBLE FOR RAT-STATS?

Several individuals have been involved in the development of this package. Key members forthis edition are:

Janet Fowler, PhD - Statistician, Office of Audit ServicesAl Kvanli, PhD - Associate Professor, Department of Business Computer Information Systems, College of Business Administration, University of North Texas

WHOM DO I CONTACT ABOUT RAT-STATS?

If you have questions about the software or suggestions for improvements, you may contactOffice of Inspector General, Office of Public Affairs at [email protected].

WHY IS IT CALLED RAT-STATS?

This is the most common question asked about the package. The software was initiallydeveloped by the Regional Advanced Techniques Staff (RATS) in San Francisco. After aninformal naming contest in the RATS office, the name RAT-STATS won by a process ofelimination.

RAT-STATS User’s Guide RAT-STATS

(Rev. 11/2005) Page 1-1

RANDOM NUMBERSATTRIBUTE APPRAISALSVARIABLE APPRAISALS


Windows Version of RAT-STATS

Opening Screen

The opening screen for the windows version of RAT-STATS is shown below. Click on HELPto view the RAT-STATS Help file and EXIT to exit the program.

RAT-STATS RAT-STATS User’s Guide

Page 1-2 (Rev. 11/2005)

SINGLE-STAGE NUMBERS

SETS OF TWO NUMBERS

SETS OF THREE NUMBERS

SETS OF FOUR NUMBERS

FRAMES - SINGLE STAGE

FRAMES - SETS OF TWO

RHC SAMPLE SELECTION

Note: RAT-STATS has been designed to ignore invalid keystrokes in all the Random Number modules, such as a minus sign or period when entering a universe size.

Random Numbers

The RAT-STATS package contains a random number generator that should be used to randomlyselect items for review. Section two of this guide explains how to use the random numbergenerator programs.


(Rev. 11/2005) Page 1-3

UNRESTRICTED

STRATIFIED

TWO-STAGE UNRESTRICTED

THREE-STAGE UNRESTRICTED

RHC TWO-STAGE

RHC THREE-STAGE

STRATIFIED CLUSTER

STRATIFIED MULTISTAGE

Note: RAT-STATS has been designed to ignore invalid keystrokes in all the Attribute Appraisal modules, such as a minus sign or period when entering a universe size.

Attribute Appraisals

Attribute appraisals allow a user to estimate the rate of occurrence of a given condition. Theuser may, for example, want to know the rate of occurrence of checks issued for less than $100. After drawing a random sample and evaluating the items selected, an attribute appraisal wouldbe used not only to estimate the rate of occurrence, but also to determine (with a measureddegree of confidence) the boundaries of the estimate.

By selecting ATTRIBUTE APPRAISALS, the user will see a window appear on the screenwith the attribute appraisal programs. Section three of this guide explains each of the attributeappraisal programs.

RAT-STATS RAT-STATS User’s Guide

Page 1-4 (Rev. 11/2005)

UNRESTRICTED

STRATIFIED


THREE STAGE UNRESTRICTED

RHC TWO-STAGE

RHC THREE-STAGE

STRATIFIED CLUSTER


POST STRATIFICATION

UNKNOWN UNIVERSE SIZE

Note: RAT-STATS has been designed to ignore invalid keystrokes in all the Variable Appraisal modules, such as a minus sign or period when entering a universe size.

Variable Appraisals

The purpose of using variable appraisals is to measure a quantitative characteristic or set ofcharacteristics. The user may, for example, want to know the value of all checks approved by acertain supervisor. After drawing a random sample and identifying the checks approved by thesupervisor, a variable appraisal would be used not only to estimate the total value, but also todetermine (with a measured degree of confidence) the boundaries of the estimate.

By selecting VARIABLE APPRAISALS, the user will see a window appear on the screen withthe variable appraisal programs. Section four of this guide explains each of the variableappraisal programs.


(Rev. 11/2005) Page 1-1

VARIABLE SAMPLE SIZE DETERMINATION - - Unrestricted - Using a Probe Sample - - Unrestricted - Using Estimated Error Rate - - Stratified

ATTRIBUTE SAMPLE SIZE DETERMINATION

Note: RAT-STATS has been designed to ignore invalid keystrokes in all the Sample Size modules, such as a minus sign or period when entering a universe size.

Sample Size Determination

The Variable Sample Size Determination program allows the user to estimate sample sizes forspecified precision percentages and specified confidence levels. In the Variable Unrestricted(Using a Probe Sample) module, the user will have the option of having the program read aprobe sample file to obtain an estimate of the universe mean and standard deviation or inputthese two estimates directly without reading a probe sample file. The Variable Unrestricted(Using Estimated Error Rate) module determines an optimum sample size by first estimating themean and standard deviation of the difference amounts using the reported amounts and anestimated error rate. The Variable Stratified module will determine optimum sample sizes forsituations where the total sample size is either predetermined or unknown.

The Attribute Sample Size Determination program determines the sample size for an attributesimple random sample. The sample size is determined to provide for a specified degree ofprecision (using the desired width of the confidence interval) at four levels of confidence (80%,90%, 95%, and 99%). The resulting sample sizes are the smallest sample sizes capable ofmeeting the specified precision requirement at the stated confidence level.

By selecting SAMPLE SIZE DETERMINATION, the user will see a window appear with thesample size determination programs. Section five of this guide explains each of the sample sizedetermination programs.

Random Numbers

RAT-STATS User’s Guide RANDOM NUMBERS - OVERVIEW

(Rev. 11/2005) Page 2-1


SETS OF TWO NUMBERS






OVERVIEW

Selecting items for a sample usually requires assigning a unique identifier to each item in theuniverse. Statistical textbooks typically assign a numbering sequence from one to the size of thesampling frame for their examples and problems. However, in the real world of sampling, theperson drawing the sample may not find such a numbering sequence and the size of the framemay not make it feasible to manually create such a sequencing technique.

Using books of random digits may cause further delays in selecting the sample items. If, forexample, the universe is numbered from 1 to 3,000, approximately 70% of the 4-digit numbersdrawn would have to be rejected since they fall outside the universe boundaries. In addition, theperson drawing the sample would probably want to sample without replacement. By selectingthis approach, the person must eliminate any duplicate selections of random numbers. This isusually accomplished by sorting the random numbers in ascending order and identifyingduplicates in the process. The sorting of the random numbers process will usually speed up thesample selection process.

This package attempts to minimize the efforts of the user in actually identifying the items to besampled. Once the user has identified the boundaries of the sampling frame, the random numbermodule will ensure that only an unduplicated list of random numbers within the stated range isselected. The random number module also allows the user to have all or a portion of the random

RANDOM NUMBERS - OVERVIEW RAT-STATS User’s Guide

Page 2-2 (Rev. 11/2005)

numbers sorted in ascending order to allow for a more efficient retrieval of the sampled items. The user also has the option of outputting the random numbers to any combination of thefollowing output formats: Printer, text file, Access table, Excel spreadsheet, or flat file.

Single-Stage Numbers

This module is used when the sampling frame of items has a numbering scheme. The framecould be a computer listing with each item numbered from one to the size of the sampling frame. The numbering scheme could also be based on check or voucher numbers assigned to eachdocument.

Sets of Two Numbers

This module is used when sample items can be easily identified in a two-step process. Forexample, the user has a computer printout of a universe that contains page numbers and no itemnumbers. The user could use this module and enter the range of page numbers (e.g., 1 to 150)and then enter the range of the maximum lines on a page (e.g., 1 to 66). This approach might bemore expedient to a user than to number all of the items in the listing.

Sets of Three Numbers

This module is used when sample items can be easily identified in a three-step process. Forexample, the user is sampling from monthly listings of transactions for a 1-year period of time. The first set of numbers entered would be for the range of months (e.g., 1 to 12). The second setof numbers entered would be for the range of the maximum number of pages for one monthlylisting (e.g., 1 to 842). The third set of numbers would be for the range of the maximum lines ona page (e.g., 1 to 66).

Sets of Four Numbers

This module would be used in situation similar to the “Sets of Three Numbers” with theadditional level of selection. For example, the user might have multiple years to select itemsfrom and might use a year, month, page, line approach to draw the sample items.

Frames - Single Stage

The user may find a situation where the universe of items has large gaps in the numbering or thenumbering scheme is reset at the beginning of each period. For example, the user may beinterested in looking at a universe of 712 checks that are numbered 1,201 through 1,483 and2,833 through 3,261. The module allows the user to enter in these two range sets (frames). Themodule will calculate the universe (e.g., 712) and for each random number selected (e.g., 10),determine the set each random selection is in (e.g., 1) and the number within that set (e.g.,1,210).

RAT-STATS User’s Guide RANDOM NUMBERS - OVERVIEW

(Rev. 11/2005) Page 2-3

The user may also have a situation where vouchers are numbered starting with 1 at the beginningof each month. For a 6-month review, the user would have six frames of voucher numbers. Therange for each month would be entered and the module would calculate the overall universe ofitems. The output would display the random number drawn and the month and voucheridentified by the random number.

Frames - Sets of Two

Similar to “Frames - Single Stage,” the user may have a situation where the universe has gaps orrepeats on a periodic basis. In addition, the user may see that a two-step process (e.g., page andline number) is the most efficient way to select the sample items. For example, the user may beworking with monthly computer listings. The number of pages may vary significantly frommonth to month. By using this module, the user could establish the frame (e.g., pages) for eachmonth and then indicate the maximum number of lines on a page (e.g., 66). If the user foundthat the maximum number of lines changed significantly from month to month, then the user hasthe option of entering the maximum boundaries (e.g., lines) for each frame (e.g., pages).

RHC Sample Selection

In certain situations a user may want to draw a multistage sample with the probability greater forselecting “larger” units in the universe. For example, a user may want to take an inventory ofitems at various warehouses. However, the user may want the larger warehouses to have agreater chance for selection. The statistical methodology developed by Rao, Hartley, andCochran allows the user to weight (e.g., square footage at each warehouse) the primary (andsecondary, if it is a three-stage sample) units and thereby increase the chance for larger units tobe sampled. This sampling methodology requires that the appropriate appraisal software beused.

SINGLE-STAGE NUMBERS RAT-STATS User’s Guide

Page 2-4 (Rev. 11/2005)

NOTE: Example is for illustrative purposes only. The sample size maynot conform to the organization’s minimum sample size standards.


Purpose

This program will generate an unduplicated quantity of random numbers (maximum of 2,000)for the user. The quantity of random numbers requested must be less than the size of thesampling frame.

Input Screen

RAT-STATS User’s Guide SINGLE-STAGE NUMBERS

(Rev. 11/2005) Page 2-5

Do you want to enter a seed number?

The program allows a seed number to be entered by the user to start the random numbergeneration. If no number is entered, then the program will use an algorithm to generate its ownseed number. The seed number algorithm is based on the clock in the computer. If the computerclock is not functioning, the user should obtain and document a seed number from anothersource (e.g., a book of random numbers). The seed number will be printed as part of the output.

Name of the audit/review:

The user may enter a brief description to document the purpose of the run. The user’s responsewill be placed at the top of each printed page. The description should be less than 40 charactersin length and may include commas and spaces.

Enter the quantity of numbers to be generated in:Sequential Order

The quantity of random numbers to be generated in sequential order should be entered in thisbox. After the quantity indicated has been generated by the program, the random numbers willbe sorted and the output will be arranged in ascending order to assist the user in retrieving thesample items. The order of selection will be printed with the random numbers. If the quantitydesired is zero, then this box can be left blank or a “0” (zero) can be entered.

Spares in Random Order

The quantity of numbers to be generated in random order should be entered in this box. Therandom numbers will be displayed in the order selected. If the quantity desired is zero, then thisbox can be left blank or a “0” (zero) can be entered.

Entering the sampling frame

The low and high numbers in the sampling frame are the boundaries of the frame from which theuser will be sampling. If the frame is a computer listing numbered 1 through 1,000 then the lowentry will be 1 and the high entry will be 1,000. If the frame is a check register with checksnumbered between 1,346 and 2,785, then the low will be 1,346 and the high will be 2,785.


Page 2-6 (Rev. 11/2005)

Important NoticeVersions of Windows RAT-STATS prior to RAT-STATS2005 used executable DOS modules when generating randomnumbers. Consequently, the random numbers generatedagreed with those obtained using DOS versions ofRAT-STATS. RAT-STATS 2005 does not utilize these DOSmodules but instead uses the full 32-bit technology of VisualBasic. As a result, random numbers generated usingRAT-STATS 2005 and later (which includes RAT-STATS2007) will not agree with those obtained using any of theprevious versions..

Output Options

The program allows for five output options. The user may select the output be sent to printer,text file, Access table, Excel spreadsheet, or flat file. The program always concludes with asummary on the screen.

Program Output

The program output can be to any combination of the above five formats. To select one or moreoutput formats, select the corresponding device/file in the above OUTPUT TO list. If Text File,Access File, Excel File, or Flat File is selected, the following message will appear:

Output to a Text File

When all the desired output formats have been selected in the OUTPUT TO box, click on thebutton labeled File Name(s). If the OUTPUT TO Text File is selected, the standard Windows“Save” file screen will appear. Type in the name of the file in the File name box and click onSave. The output file will be saved with a “.TXT” extension (e.g., C:\TEMP\OUTDISK.TXT). After saving the file, the program will return to the original input screen for this module.

The output shown next is file C:\ TEMP\OUTDISK.TXT and is the result of generating10 random numbers between 1 and 1,000 along with 4 spares. For each random numbergenerated, two pieces of information are provided. The first is the order of selection. Thisnumber indicates the order in which the random number was generated by the program. Thesecond piece of information is the actual random number selected.


(Rev. 11/2005) Page 2-7


Department of Health and Human Services OIG - Office of Audit ServicesDate: 10/12/2004 Random Number Generator Time: 13:34 AUDIT: User Guide Example

SEED NUMBER: 48863.78 FRAME SIZE: 1,000

FILE OF RANDOM NUMBERS: C:\TEMP\OUTDISK.TXT


THE NUMBERS ARE IN THE FOLLOWING FORMAT IN YOUR FILE: POSITIONS 1 THROUGH 6 - ORDER OF SELECTION POSITIONS 7 THROUGH 17 - RANDOM NUMBEREACH COLUMN OF NUMBERS IS RIGHT JUSTIFIED.

Selection Order Value 10 22 μ The 10 random values start here. 8 141 4 236 3 247 7 257 9 301 2 643 6 650 1 718 5 821 11 980 μ The 4 spare values start here. 12 507 13 440 14 557

SUMMATION OF RANDOM NUMBERS = 6,520


Page 2-8 (Rev. 11/2005)

Output to an Access Database

When all the desired output formats have been selected in the OUTPUT TO box, click on thebutton labeled File Name(s). If the OUTPUT TO Access File is selected, the standard Windows“Save” file screen will appear. Fill in the name of the Access database in the File name box. The output file will be saved with the “.MDB” extension (e.g., C:\TEMP\OUTACCESS.MDB). By clicking on the Save button, the program will return to the original input screen for thismodule. After clicking on Continue, the user will see the following form:

The program is asking for the name of the Access table to create in the specified database(C:\TEMP\OUTACCESS.MDB for this illustration). The table name “OutSingle” will be used. The program will then create two tables (OutSingle-Values and OutSingle-Spares) withindatabase C:\TEMP\OUTACCESS.MDB.

The tables OutSingle-Values and OutSingles-Spares, shown next, are the result of generating10 random numbers between 1 and 1,000 along with 4 spares. For each random numbergenerated, two pieces of information are created. The first is the order of selection. This numberindicates the order in which the random number was generated by the program. The secondpiece of information is the actual random number selected. The seed number used by therandom number generator is in the far-right column.


(Rev. 11/2005) Page 2-9

NOTE: Example is for illustrative purposesonly. The sample size may not conform tothe organization’s minimum sample sizestandards.

The OutSingle-Values table contains ten random numbers, the seed number, the date, and thetime.

The OutSingle-Spares table contains four spare values.

Output to an Excel Spreadsheet

When all the desired output formats have been selected in the OUTPUT TO box, click on thebutton labeled File Name(s). If the OUTPUT TO Excel File is selected, the standard Windows“Save” file screen will appear. Fill in the name of the Excel spreadsheet in the File name box. The output file will be saved with the “.XLS” extension (e.g., C:\TEMP\OUTEXCEL.XLS ).

The following output will be contained in the Excel spreadsheet C:\TEMP\OUTEXCEL.XLS. By clicking on the tab labeled “SPARES,” the user will obtain the output immediately following. For each random number generated, two pieces of information are created. The first is the orderof selection. This number indicates the order in which the random number was generated by theprogram. The second piece of information is the actual random number selected. The outputalso contains the program execution date and time, the name of the audit/review, the seednumber, and the size of the frame.


Page 2-10 (Rev. 11/2005)

NOTE: Example is for illustrative purposesonly. The sample size may not conform tothe organization’s minimum sample sizestandards.

These are the ten random values in the Excel spreadsheet named “VALUES” located inC:\TEMP\OUTEXCEL.XLS.

These are the four spares in the Excel spreadsheet named “SPARES.”

NOTE: The user must first exit RAT-STATS in order to view this file.

Output to a Flat File

When all the desired output formats have been selected in the OUTPUT TO box, click on thebutton labeled File Name(s). If the OUTPUT TO Flat File is selected, the standard Windows“Save” file screen will appear. Fill in the name of the file in the File name box. The name ofthe file for this illustration is C:\TEMP\OUTFLAT.

The following output is the text file created by selecting “Flat File” as one of the output options. This file is the output file created when generating 10 random numbers between 1 and 1,000along with 4 spares. Notice that the order of selection and the random values contain leading


(Rev. 11/2005) Page 2-11


zero values. This file is often useful as an input file for selecting random records using amainframe computer.



FILE OF RANDOM NUMBERS: C:\TEMP\OUTFLAT


00100000000022 μ The 10 random values start here.00080000000141000400000002360003000000024700070000000257000900000003010002000000064300060000000650000100000007180005000000082100110000000980 μ The 4 spare values start here.001200000005070013000000044000140000000557

SUMMATION OF RANDOM NUMBERS = 6,520


Page 2-12 (Rev. 11/2005)

NOTE: Example is for illustrative purposes only. The sample size maynot conform with the organization’s minimum sample size standards.


DATE: 10/12/2004 RANDOM NUMBER GENERATOR TIME: 13:34AUDIT: User Guide Example


10 RANDOM NUMBERS IN SEQUENTIAL ORDER

ORDER OF RANDOM ORDER OF RANDOM ORDER OF RANDOMSELECTION NUMBER SELECTION NUMBER SELECTION NUMBER

10 - 22 8- 141 4 - 236 3 - 247 7 - 257 9 - 301 2 - 643 6 - 650 1 - 718 5 - 821

====================NEW PAGE====================




4 RANDOM NUMBERS IN GENERATED ORDER

ORDER OF RANDOM ORDER OF RANDOM ORDER OF RANDOMSELECTION NUMBER SELECTION NUMBER SELECTION NUMBER 11 - 980 12 - 507 13 - 440 14 - 557

SUMMATION OF RANDOM NUMBERS 6,520

Output to a Printer

If the user selects the printer for output, the standard Windows “Print” dialog box will appear. Select the appropriate printer and click on OK. The output immediately following is the printeroutput when generating 10 random numbers between 1 and 1,000 along with 4 spares. For eachrandom number generated, two pieces of information are displayed. The first is the order ofselection. This number indicates the order in which the random number was generated by theprogram. The second piece of information is the actual random number selected. Thesubsequent random numbers are in sequential order going from left to right across the page. Each line will have a maximum of three random numbers with the last line having three or lessdepending on the quantity requested.

Output to Screen


(Rev. 11/2005) Page 2-13


The program always concludes with a summary on the screen. The following screen is thesummary for this illustration:

SETS OF TWO NUMBERS RAT-STATS User’s Guide

Page 2-14 (Rev. 11/2005)

SETS OF TWO NUMBERS

Purpose

This program will generate an unduplicated pair (called a set) of random numbers (maximum of2,000 pairs) for the user. This program may be more efficient for a user when a sample item canbe quickly identified through a two-step process (e.g., page and line number).

Input Screen

RAT-STATS User’s Guide SETS OF TWO NUMBERS

(Rev. 11/2005) Page 2-15





Enter the quantity of numbers to be generated in:sequential order

The quantity of random numbers generated in sequential order should be entered in this box. After the quantity indicated has been generated by the program, the random numbers will besorted and printed in ascending order to assist the user in retrieving the sample items. The orderof selection will be printed with the random numbers. If the quantity desired is zero, then thisbox can be left blank or a “0” (zero) can be entered.

Spares in random order

The quantity of numbers to be generated in random order should be entered in this box. Therandom numbers will be printed in the order selected. If the quantity desired is zero, then thisbox can be left blank or a “0” (zero) can be entered.


The program prompts the user to enter the low and high numbers of the sampling frame thatconstitute the first range in the pair of ranges for the entire frame. If, for example, the user was


Page 2-16 (Rev. 11/2005)

Important NoticeVersions of Windows RAT-STATS prior to RAT-STATS2005 used executable DOS modules when generating randomnumbers. Consequently, the random numbers generatedagreed with those obtained using DOS versions ofRAT-STATS. RAT-STATS 2005 does not utilize these DOSmodules but instead uses the full 32-bit technology of VisualBasic. As a result, random numbers generated usingRAT-STATS 2005 and later (which includes RAT-STATS2007) will not agree with those obtained using any of theprevious versions.

planning to select items from a computer printout that had pages numbered 1 through 658 andhad 66 lines on each page, then the low for the first set (pages) would be 1 and the high for thefirst set would be 658. The low for the second set (lines on a page) would be 1 and the high forthe second set would be 66. The overall frame size for this sample would be 43,428 (658 times66). For ease of use, the path to the sample items should dictate the selection of frameboundaries that are placed in each set. Thus, for the above example, the user should give thepage boundaries for the first set and the line boundaries for the second set. While doing this inreverse order (line boundaries first) is permissible, the sequential ordering of the pairs ofnumbers would be in ascending order by line instead of by page. Such ordering would normallyincrease the time required by the user to locate and select the sample items.

Output Options

The program allows for five output options. The user may select the output be sent to printer,text file, Access table, Excel spreadsheet, or flat file. The user selects the appropriate output. The program always concludes with a summary on the screen.

Program Output



(Rev. 11/2005) Page 2-17

NOTE: Example is for illustrative purposes only. The sample size may not conform to theorganization’s minimum sample size standards.


When all the desired output formats have been selected in the OUTPUT TO box, click on thebutton labeled File Name(s). If the OUTPUT TO Text File is selected, the standard Windows“Save” file screen will appear. Enter the name of the file in the File name box. A text file willbe saved with a “.TXT” extension (e.g., C:\TEMP\OUTDISK.TXT). By clicking on the Savebutton, the program will return to the original input screen for this module.

The output shown below is file C:\ TEMP\OUTDISK.TXT and is the output using the values inthe preceding Entering the sampling frame display. Ten random values were selected alongwith four spares. For each set of random numbers generated, three pieces of information areprovided. The first is the order of selection. This number indicates the order in which therandom number was generated by the program. The next piece of information is the randomnumber generated from the first set of frame boundaries. The third number is the randomnumber generated from the second set of boundaries.





THE NUMBERS ARE IN THE FOLLOWING FORMAT IN YOUR FILE: POSITIONS 1 THROUGH 6 - ORDER OF SELECTION POSITIONS 7 THROUGH 17 - FIRST NUMBER OF SET POSITIONS 18 THROUGH 30 - SECOND NUMBER OF SETEACH COLUMN OF NUMBERS IS RIGHT JUSTIFIED.

Selection First Second Order Number Number 9 147 47 μ The 10 random values start here. 2 165 26 6 169 6 4 200 31 8 250 56 1 309 41 5 471 50 7 491 48 3 510 29 10 598 32 11 48 42 μ The 4 spare values start here. 12 101 56 13 327 5 14 288 30


Page 2-18 (Rev. 11/2005)


When all the desired output formats have been selected in the OUTPUT TO box, click on thebutton labeled File Name(s). If the OUTPUT TO Access File is selected, the standard Windows“Save” file screen will appear. Enter the name of the Access database in the File name box. The output file will be saved with the “.MDB” extension (e.g., C:\TEMP\OUTACCESS.MDB). By clicking on the Save button, the program will return to the original input screen for thismodule. After clicking on Continue, the user will see the following form:

The program is asking for the name of the Access table to create in the specified database(C:\TEMP\OUTACCESS.MDB for this illustration). The table name “Out2Stage” will be used. The program will then create two tables (Out2Stage-Values and Out2Stage-Spares) withindatabase C:\TEMP\OUTACCESS.MDB.

The tables Out2Stage-Values and Out2Stage-Spares, shown next, are the result of generating10 random numbers along with 4 spares using the values in the preceding Entering thesampling frame display. For each set of random numbers generated, three pieces of informationare created. The first is the order of selection. This number indicates the order in which therandom number was generated by the program. The next piece of information is the randomnumber generated from the first set of frame boundaries. The third number is the randomnumber generated from the second set of boundaries. The seed number used by the randomnumber generator is in the far-right column. The Out2Stage-Values table contains 10 randomnumbers, the seed number, the date, and the time.


(Rev. 11/2005) Page 2-19


The Out2Stage-Spares table contains four spare values.


When all the desired output formats have been selected in the OUTPUT TO box, click on thebutton labeled File Name(s). If the OUTPUT TO Excel File is selected, the standard Windows“Save” file screen will appear. Enter the name of the Excel spreadsheet in the File name box. The output file will be saved with the “.XLS” extension (e.g., C:\TEMP\OUTEXCEL.XLS).

The following output will be contained in the Excel spreadsheet C:\TEMP\OUTEXCEL.XLSand is the result of generating 10 random numbers along with 4 spares using the values in thepreceding Entering the sampling frame display. By clicking on the tab labeled “SPARES,” theuser will obtain the output immediately following. For each set of random numbers generated,


Page 2-20 (Rev. 11/2005)


three pieces of information are created. The first is the order of selection. This number indicatesthe order in which the random number was generated by the program. The next piece ofinformation is the random number generated from the first set of frame boundaries. The thirdnumber is the random number generated from the second set of boundaries. The output alsocontains the program execution date and time, the name of the audit/review, the seed number,and the size of the frame.

These are the 10 values in the Excel spreadsheet named “VALUES” located inC:\TEMP\OUTEXCEL.XLS.

These are the four spares in the Excel spreadsheet named “SPARES.”

NOTE: The user must first exit RAT-STATS inorder to view this file.


(Rev. 11/2005) Page 2-21



When all the desired output formats have been selected in the OUTPUT TO box, click on thebutton labeled File Name(s). If the OUTPUT TO Flat File is selected, the standard Windows“Save” file form will appear. Fill in the name of the file in the File name box. The name of thefile for this illustration is C:\TEMP\OUTFLAT.

The output immediately following is the text file created by selecting “Flat File” as one of theoutput options. This output is the file created when generating 10 random numbers along with 4spares using the values in the preceding Entering the sampling frame display. For each set ofrandom numbers generated, three pieces of information are printed. The first is the order ofselection. This number indicates the order in which the random number was generated by theprogram. The next piece of information is the random number generated from the first set offrame boundaries. The third number is the random number generated from the second set ofboundaries. Notice that the order of selection and the random values contain leading zerovalues. This file is often useful as an input file for selecting random records using a mainframecomputer.





000900000001470000000047 μ The 10 random values start here.000200000001650000000026000600000001690000000006000400000002000000000031000800000002500000000056000100000003090000000041000500000004710000000050000700000004910000000048000300000005100000000029001000000005980000000032001100000000480000000042 μ The 4 spare values start here.001200000001010000000056001300000003270000000005001400000002880000000030


Page 2-22 (Rev. 11/2005)

Output to a Printer

If the user selects the printer for output, the standard Windows “Printer” dialog box will appear. Select the appropriate printer and click on OK.

The output immediately following is the printer output when generating 10 random numbersalong with 4 spares using the values in the preceding Entering the sampling frame display. For each set of random numbers generated, three pieces of information are printed. The first isthe order of selection. This number indicates the order in which the random number wasgenerated by the program. The next piece of information is the random number generated fromthe first set of frame boundaries. The third number is the random number generated from thesecond set of boundaries. For numbers printed in sequential order, the lowest set of numbers isprinted first. The subsequent sets of random numbers are in sequential order going from left toright across the page. Each line will have a maximum of two sets of random numbers with thelast line having one or two sets depending on the quantity requested.

The random sets of numbers in generated order will begin on a new page after all the sets insequential order have been printed.


(Rev. 11/2005) Page 2-23





ORDER OF ORDER OF SELECTION FIRST NBR. SECOND NBR. SELECTION FIRST NBR. SECOND NBR.

9 - 147 47 2 - 165 26 6 - 169 6 4 - 200 31 8 - 250 56 1 - 309 41 5 - 471 50 7 - 491 48 3 - 510 29 10 - 598 32

====================NEW PAGE====================





ORDER OF ORDER OF SELECTION FIRST NBR. SECOND NBR. SELECTION FIRST NBR. SECOND NBR.

11 - 48 42 12 - 101 56 13 - 327 5 14 - 288 30



Page 2-24 (Rev. 11/2005)


Output to Screen


RAT-STATS User’s Guide SETS OF THREE NUMBERS

(Rev. 11/2005) Page 2-25


Purpose

This program will generate an unduplicated set of random numbers (maximum of 2,000 sets) forthe user. This program may be more efficient for a user when a sample item can be quicklyidentified through a three-step process (e.g., month, page and line number).

Input Screen

SETS OF THREE NUMBERS RAT-STATS User’s Guide

Page 2-26 (Rev. 11/2005)










The low and high numbers in the frame are the boundaries of the frame from which the user will


(Rev. 11/2005) Page 2-27

Important NoticeVersions of Windows RAT-STATS prior to RAT-STATS2005 used executable DOS modules when generatingrandom numbers. Consequently, the random numbersgenerated agreed with those obtained using DOS versions ofRAT-STATS. RAT-STATS 2005 does not utilize theseDOS modules but instead uses the full 32-bit technology ofVisual Basic. As a result, random numbers generated usingRAT-STATS 2005 and later (which includes RAT-STATS2007) will not agree with those obtained using any of theprevious versions.

be sampling. If, for example, the user was planning to select items from a year’s worth ofcomputer printouts that had pages numbered 1 through 658 each month and had 66 lines on eachpage, then the first set would be months; the second set would be pages; and the third set wouldbe lines. For the first set the low would be 1 and the high would be 12. The low for the secondset would be 1 and the high would be 658. The low for the third set would be 1 and the highwould be 66. The overall frame size for this sample would be 521,136 (12 times 658 times 66).

For ease of use, the path to the sample items should dictate the order of frame boundaries that areplaced in each set. Thus, for the above example, the user should give the month boundaries forthe first set, page boundaries for the second set and the line boundaries for the third set. Whiledoing this in reverse order (line boundaries first) is permissible, the sequential ordering of thesets of numbers would be in ascending order by line instead of by page or month. Such orderingwould normally increase the time required by the user to locate and select the sample items.

Output Options

The program allows for five output options. The user may select the output to be sent to printer,text file, Access database, Excel spreadsheet, or flat file. The user selects the appropriate output. The program always concludes with a summary on the screen.

Program Output



Page 2-28 (Rev. 11/2005)

NOTE: Example isfor illustrativepurposes only. Thesample size may notconform to theorganization’sminimum samplesize standards.



The output shown below is file C:\ TEMP\OUTDISK.TXT and is the output using the values inthe preceding Entering the sampling frame display. Four random values were selected alongwith two spares. For each set of random numbers generated, four pieces of information areprovided. The first is the order of selection. This number indicates the order in which therandom number was generated by the program. The next piece of information is the randomnumber generated from the first set of frame boundaries. The third piece of information is therandom number generated from the second set of boundaries. The fourth piece of information isthe random number generated from the third set of boundaries.





THE NUMBERS ARE IN THE FOLLOWING FORMAT IN YOURFILE: POSITIONS 1 THROUGH 6 - ORDER OF SELECTION POSITIONS 7 THROUGH 17 - FIRST NUMBER OF SET POSITIONS 18 THROUGH 30 - SECOND NUMBER OF SET POSITIONS 31 THROUGH 43 - THIRD NUMBER OF SETEACH COLUMN OF NUMBERS IS RIGHT JUSTIFIED.

Selection First Second Third Order Number Number Number

1 3 484 54 μ The 4 random values start here 2 9 606 27 3 10 192 19 4 12 513 59

5 6 574 56 μ The 2 spare values start here 6 2 615 52


(Rev. 11/2005) Page 2-29



The program is asking for the name of the Access table to create in the specified database(C:\TEMP\OUTACCESS for this illustration). The table name “Out3Stage” will be used. Theprogram will then create two tables (Out3Stage-Values and Out3Stage-Spares) within databaseC:\TEMP\OUTACCESS.

The table Out3Stage-Values shown below is the result of generating four random numbers alongwith two spares using the values in the preceding Entering the sampling frame display. Foreach set of random numbers generated, four pieces of information are created. The first is theorder of selection. This number indicates the order in which the random number was generatedby the program. The next piece of information is the random number generated from the first setof frame boundaries. The third number is the random number generated from the second set ofboundaries. The fourth piece of information is the random number generated from the third setof boundaries. The seed number used by the random number generator is in the far-rightcolumn. The Out3Stage-Values table contains four random numbers, the seed number, the date,and the time.


Page 2-30 (Rev. 11/2005)


The Out3Stage-Spares table containing the two spare values is shown below.



The following output will be contained in the Excel spreadsheet C:\TEMP\OUTEXCEL.XLSand is the result of generating four random numbers along with two spares using the values inthe preceding Entering the sampling frame display. By clicking on the tab labeled “SPARES,”the user will obtain the spreadsheet containing the spare values. For each set of random numbersgenerated, four pieces of information are printed. The first is the order of selection. Thisnumber indicates the order in which the random number was generated by the program. Thenext piece of information is the random number generated from the first set of frame boundaries. The third number is the random number generated from the second set of boundaries. The fourthnumber is the random number generated from the third set of boundaries. The output alsocontains the program execution date and time, the name of the audit/review, the seed number,and the size of the frame.

These are the four values in the Excel spreadsheet named “VALUES” located inC:\TEMP\OUTEXCEL.XLS:


(Rev. 11/2005) Page 2-31


These are the two spares in theExcel spreadsheet named“SPARES.”



The output immediately following is the text file created by selecting “Flat File” as one of theoutput options. This output is the file created when generating four random numbers along withtwo spares using the values in the preceding Entering the sampling frame display. For each setof random numbers generated, four pieces of information are printed. The first is the order ofselection. This number indicates the order in which the random number was generated by theprogram. The next piece of information is the random number generated from the first set offrame boundaries. The third number is the random number generated from the second set ofboundaries. The fourth number is the random number generated from the third set of boundaries. Notice that the order of selection and the random values contain leading zero values. This file isoften useful as an input file for selecting random records using a mainframe computer.


Page 2-32 (Rev. 11/2005)






0001000000000300000004840000000054 μ The 4 random values start here000200000000090000000606000000002700030000000010000000019200000000190004000000001200000005130000000059

0005000000000600000005740000000056 μ The 2 spare values start here0006000000000200000006150000000052

Output to a Printer


The output immediately following is the printer output when generating four random numbersalong with two spares using the values in the preceding Entering the sampling frame display. For each set of random numbers generated, four pieces of information are printed. The first isthe order of selection. This number indicates the order in which the random number wasgenerated by the program. The next piece of information is the random number generated fromthe first set of frame boundaries. The third number is the random number generated from thesecond set of boundaries. The fourth number is the random number generated from the third setof boundaries.



(Rev. 11/2005) Page 2-33





ORDER OF SELECTION FIRST NBR. SECOND NBR. THIRD NBR.

1 - 3 484 54 2 - 9 606 27 3 - 10 192 19 4 - 12 513 59

=====================NEW PAGE===================





ORDER OF SELECTION FIRST NBR. SECOND NBR. THIRD NBR.

5 - 6 574 56 6 - 2 615 52



Page 2-34 (Rev. 11/2005)


Output to Screen


RAT-STATS User’s Guide SETS OF FOUR NUMBERS

(Rev. 11/2005) Page 2-35


Purpose

This program will generate an unduplicated set of random numbers (maximum of 2,000 sets) forthe user. This program may be more efficient for a user when a sample item can be quicklyidentified through a four-step process (e.g., year, month, page, and line number).

Input Screen

SETS OF FOUR NUMBERS RAT-STATS User’s Guide

Page 2-36 (Rev. 11/2005)











(Rev. 11/2005) Page 2-37

Important NoticeVersions of Windows RAT-STATS prior to RAT-STATS 2005used executable DOS modules when generating randomnumbers. Consequently, the random numbers generated agreedwith those obtained using DOS versions of RAT-STATS. RAT-STATS 2005 does not utilize these DOS modules butinstead uses the full 32-bit technology of Visual Basic. As aresult, random numbers generated using RAT-STATS 2005 andlater (which includes RAT-STATS 2007) will not agree withthose obtained using any of the previous versions.

The program prompts the user to enter the low and high numbers for each of the four sets offrames. If, for example, the user was planning to select items from five years’ worth of computerprintouts that had pages numbered 1 through 658 each month and had 66 lines on each page,then the first set would be year, the second set would be month, the third set would be pages, andthe fourth set would be lines. For the first set the low number would be 1 and the high would be5. The low for the second set would be 1 and the high would be 12. The low for the third setwould be 1 and the high would be 658. The low for the fourth set would be 1 and the high wouldbe 66. The overall frame size for this sample would be 2,605,680 (5 times 12 times 658 times66).

For ease of use, the path to the sample items should dictate the order of frame boundaries that areplaced in each set. Thus, for the above example, the user should give the year boundaries for thefirst set, month boundaries for the second set, page boundaries for the third set, and lineboundaries for the fourth set. While doing this in reverse order (line boundaries first) ispermissible, the sequential ordering of the sets of numbers would be in ascending order by lineinstead of by page, month, or year. Such ordering would normally increase the time required bythe user to locate and select the sample items.

Output Options

The program allows for five output options. The user may select the output to be sent to printer,text file, Access database, Excel spreadsheet, or flat file. The user selects the appropriate output. The program always concludes with a summary on the screen.

Program Output

The program output can be to any combination of the above five formats. To select one or moreoutput formats, select the corresponding device/file in the above OUTPUT TO list. If Text File,Access File, Excel File, or Flat File is selected, the message on the next page will appear:


Page 2-38 (Rev. 11/2005)

NOTE: Example isfor illustrativepurposes only. Thesample size may notconform to theorganization’sminimum samplesize standards.



The output shown below is file C:\ TEMP\OUTDISK and is the output using the values in thepreceding Entering the sampling frame display. Three random values were selected alongwith two spares. For each set of random numbers generated, five pieces of information areprovided. The first is the order of selection. This number indicates the order in which therandom number was generated by the program. The next piece of information is the randomnumber generated from the first set of frame boundaries. The third piece of information is therandom number generated from the second set of boundaries. The fourth piece of information isthe random number generated from the third set of boundaries. The fifth piece of information isthe random number generated from the fourth set of boundaries.


SEED NUMBER: 35731.75 FRAME SIZE: 2,605,680



THE NUMBERS ARE IN THE FOLLOWING FORMAT IN YOUR FILE: POSITIONS 1 THROUGH 6 - ORDER OF SELECTION POSITIONS 7 THROUGH 17 - FIRST NUMBER OF SET POSITIONS 18 THROUGH 30 - SECOND NUMBER OF SET POSITIONS 31 THROUGH 43 - THIRD NUMBER OF SET POSITIONS 44 THROUGH 56 - FOURTH NUMBER OF SETEACH COLUMN OF NUMBERS IS RIGHT JUSTIFIED.


(Rev. 11/2005) Page 2-39

Selection First Second Third Fourth Order Number Number Number Number 1 1 4 114 24 μ The 3 random values 3 4 10 442 48 start here 2 4 10 501 42 4 4 5 351 55 μ The 2 spare values 5 4 9 589 20 start here



The program is asking for the name of the Access table to create in the specified database(C:\TEMP\OUTACCESS for this illustration). The table name “Out4Stage” will be used. Theprogram will then create two tables (Out4Stage-Values and Out4Stage-Spares) within databaseC:\TEMP\OUTACCESS.

The table Out4Stage-Values shown below is the result of generating four random numbers alongwith two spares using the values in the preceding Entering the sampling frame display. Foreach set of random numbers generated, five pieces of information are created. The first is theorder of selection. This number indicates the order in which the random number was generatedby the program. The next piece of information is the random number generated from the first setof frame boundaries. The third number is the random number generated from the second set ofboundaries. The fourth piece of information is the random number generated from the third setof boundaries. The fifth piece of information is the random number generated from the fourthset of boundaries. The seed number used by the random number generator is in the far-rightcolumn. The Out4Stage-Values table contains three random numbers, the seed number, the date,and the time.


Page 2-40 (Rev. 11/2005)


The table Out4Stage-Spares containing the two spare values is shown below:



The following output will be contained in the Excel spreadsheet C:\TEMP\OUTEXCEL and isthe result of generating three random numbers along with two spares using the values in thepreceding Entering the sampling frame display. By clicking on the tab labeled “SPARES,” theuser will obtain the spreadsheet containing the spare values. For each set of random numbersgenerated, five pieces of information are printed. The first is the order of selection. This numberindicates the order in which the random number was generated by the program. The next pieceof information is the random number generated from the first set of frame boundaries. The thirdnumber is the random number generated from the second set of boundaries. The fourth numberis the random number generated from the third set of boundaries. The fifth number is therandom number generated from the fourth set of boundaries. The output also contains theprogram execution date and time, the name of the audit/review, the seed number, and the size ofthe frame.


(Rev. 11/2005) Page 2-41


These are the three values in the Excel spreadsheet named “VALUES” located inC:\TEMP\OUTEXCEL.XLS:

The two spares values in the Excel spreadsheet named “SPARES” are shown below:



The output immediately following is the text file created by selecting “Flat File” as one of theoutput options. This output is the file created when generating three random numbers along withtwo spares using the values in the preceding Entering the sampling frame display. For each setof random numbers generated, five pieces of information are printed. The first is the order ofselection. This number indicates the order in which the random number was generated by theprogram. The next piece of information is the random number generated from the first set offrame boundaries. The third number is the random number generated from the second set ofboundaries. The fourth number is the random number generated from the third set of boundaries. The fifth number is the random number generated from the fourth set of boundaries. Notice that


Page 2-42 (Rev. 11/2005)


the order of selection and the random values contain leading zero values. This file is often usefulas an input file for selecting random records using a mainframe computer.





00010000000001000000000400000001140000000024 μ The 3 random values start here0003000000000400000000100000000442000000004800020000000004000000001000000005010000000042

00040000000004000000000500000003510000000055 μ The 2 spare values start here00050000000004000000000900000005890000000020

Output to a Printer


The output immediately following is the printer output when generating four random numbersalong with two spares using the values in the preceding Entering the sampling frame display. For each set of random numbers generated, five pieces of information are printed. The first isthe order of selection. This number indicates the order in which the random number wasgenerated by the program. The next piece of information is the random number generated fromthe first set of frame boundaries. The third number is the random number generated from thesecond set of boundaries. The fourth number is the random number generated from the third setof boundaries. The fifth number is the random number generated from the fourth set ofboundaries.



(Rev. 11/2005) Page 2-43





ORDER OF SELECTION FIRST NBR. SECOND NBR. THIRD NBR. FOURTH NBR.

1 - 1 4 114 24 3 - 4 10 442 48 2 - 4 10 501 42

====================NEW PAGE===================





ORDER OF SELECTION FIRST NBR. SECOND NBR. THIRD NBR. FOURTH NBR.

4 - 4 5 351 55 5 - 4 9 589 20



Page 2-44 (Rev. 11/2005)


Output to Screen


RAT-STATS User’s Guide FRAMES - SINGLE STAGE

(Rev. 11/2005) Page 2-45


Purpose

This program will generate an unduplicated set of random numbers (maximum of 2,000 sets) forthe user. This program is more efficient for a user when the universe of sample items may eithercontain gaps of numbers or the numbering system repeats within the universe. For example, theuniverse of transactions that is of interest to the user may be grouped with other transactions in acomputer printout that has all items numbered. The transactions of interest may be numberedfrom 1 through 1,050 and begin again at 8,405 and run through 9,565. Rather than selectingrandom numbers between 1 and 9,565 and not be able to use the numbers between 1,050 and8,405 or having the user renumber the subuniverse of desired transactions, this program could beused with the existing numbering system.

Input Screen

FRAMES - SINGLE STAGE RAT-STATS User’s Guide

Page 2-46 (Rev. 11/2005)






The quantity of random numbers to be generated in sequential order should be entered in thisbox. After the quantity indicated has been generated by the program, the random numbers willbe sorted and the output will be in ascending order to assist the user in retrieving the sampleitems. The order of selection will be printed with the random numbers. If the quantity desired iszero, then this box can be left blank or a “0” (zero) can be entered.



Enter the number of frames in your universe (Max = 51)

A frame is a grouping of contiguous sample items within the universe. In the illustration tofollow, the user would have two frames. The first frame would contain items 1 through 1,050and the second frame would contain items 8,405 through 9,565.

Enter or View low/high values

Based on the frame count entered by the user, the low and high boundaries for each frame mustbe entered. By clicking on the “Enter or view low/high values” button the following form


(Rev. 11/2005) Page 2-47


appears if the number of frames is entered as “2.” In general, there will be one line in this tablefor each frame in the universe.

Output Options


Program Output



Page 2-48 (Rev. 11/2005)



When all the desired output formats have been selected in the OUTPUT TO box, click on thebutton labeled File Name(s). If the OUTPUT TO Text File is selected, the standard Windows“Save” file screen will appear. Fill in the name of the file in the File name box. A text file willbe saved with a “.TXT” extension (e.g., C:\TEMP\OUTDISK.TXT). By clicking on the Savebutton, the program will return to the original input screen for this module.

The output shown below is file C:\TEMP\OUTDISK.TXT and is the result of generating threerandom numbers along with two spares using the low/high values in the earlier screen. For eachrandom number generated, four pieces of information are displayed. The first is the order ofselection. This number indicates the order in which the random number was generated by theprogram. The next piece of information is the random number generated. The next two piecesof information result in converting the random number into the appropriate frame and the itemwithin the frame.





THE NUMBERS ARE IN THE FOLLOWING FORMAT IN YOUR FILE: POSITIONS 1 THROUGH 6 - ORDER OF SELECTION POSITIONS 7 THROUGH 20 - RANDOM NUMBER POSITIONS 21 THROUGH 29 - FRAME NUMBER POSITIONS 30 THROUGH 42 - ITEM NUMBER WITHIN FRAMEEACH COLUMN OF NUMBERS IS RIGHT JUSTIFIED.

Selection Random Frame Item Order Number Number Number 3 119 1 119 μ The 3 random values start here. 1 1308 2 8662 2 1603 2 8957 4 322 1 322 μ The 2 spare values start here. 5 1553 2 8907


(Rev. 11/2005) Page 2-49



The program is asking for the name of the Access table to create in the specified database(C:\TEMP\OUTACCESS.MDB for this illustration). The table name “OutFrame1” will be used. The program will then create two tables (OutFrame1-Values and OutFrame1-Spares) withindatabase C:\TEMP\OUTACCESS.MDB.

The tables OutFrame1-Values and OutFrame1-Spares, shown next, are the result of generatingthree random numbers along with two spares using the low/high values in the earlier screen. Foreach random number generated, four pieces of information are displayed. The first is the orderof selection. This number indicates the order in which the random number was generated by theprogram. The next piece of information is the random number generated. The next two piecesof information result in converting the random number into the appropriate frame and the itemwithin the frame. The seed number used by the random number generator is in the far-rightcolumn.

The OutFrame1-Values table contains three random numbers, the seed number, the date, and thetime.


Page 2-50 (Rev. 11/2005)


The OutFrame1-Spares table contains two spare values.



The following output will be contained in the Excel spreadsheet named “VALUES” located inC:\TEMP\OUTEXCEL.XLS and is the result of generating three random numbers along withtwo spares using the low/high values in the earlier screen. By clicking on the tab labeled“SPARES” the user will obtain the spreadsheet containing the spare values.

For each random number generated, four pieces of information are displayed. The first is theorder of selection. This number indicates the order in which the random number was generatedby the program. The next piece of information is the random number generated. The next twopieces of information result in converting the random number into the appropriate frame and theitem within the frame. The output also contains the program execution date and time, the nameof the audit/review, the seed number, and the total size of the sampling frame.


(Rev. 11/2005) Page 2-51


These are the three random values in the Excel spreadsheet named “VALUES” located inC:\TEMP\OUTEXCEL.XLS.

These are the two spares in the Excel spreadsheet named “SPARES.”



The output immediately following is the text file created by selecting “Flat File” as one of theoutput options. If the OUTPUT TO Flat File is selected, the standard Windows “Save” filescreen will appear. Fill in the name of the file in the File name box. The name of the file forthis illustration is C:\TEMP\OUTFLAT.

This file is the output file when generating three random numbers along with two spares usingthe low/high values from the earlier screen. Notice that the order of selection and the randomvalues contain leading zero values. This file is often useful as an input file for selecting randomrecords using a mainframe computer.

For each random number generated, four pieces of information are displayed. The first is theorder of selection. This number indicates the order in which the random number was generated


Page 2-52 (Rev. 11/2005)


by the program. The next piece of information is the random number generated. The next twopieces of information result in converting the random number into the appropriate frame and theitem within the frame.





0003000000011900010000000119 μ The three random values start here.000100000013080002000000866200020000001603000200000089570004000000032200010000000322 μ The two spare values start here.0005000000155300020000008907

Output to a Printer

If the user selects the printer for output, the standard Windows “Print” dialog box will appear. Select the appropriate printer and click on OK.

The following printer output is the result of generating three random numbers along with twospares using the low/high values in the earlier screen. For each random number generated, fourpieces of information are displayed. The first is the order of selection. This number indicatesthe order in which the random number was generated by the program. The next piece ofinformation is the random number generated. The next two pieces of information result inconverting the random number into the appropriate frame and the item within the frame. Summary information will be displayed on the screen.

For numbers printed in sequential order of the printout, the lowest random number and thecorresponding conversion are printed first. The subsequent numbers are in sequential order withthe random number and its equivalent value after conversion on each line. The random numbersin generated order will begin on a new page after all the sequential sets have been printed.


(Rev. 11/2005) Page 2-53





ORDER OF RANDOM -CONVERTED NUMBER-SELECTION NUMBER FRAME ITEM NUMBER

3 - 119 1 1191 - 1,308 2 8,6622 - 1,603 2 8,957

=====================NEW PAGE====================





ORDER OF RANDOM -CONVERTED NUMBER-SELECTION NUMBER FRAME ITEM NUMBER

4 - 322 1 3225 - 1,553 2 8,907



Page 2-54 (Rev. 11/2005)


Output to Screen


RAT-STATS User’s Guide FRAMES - SETS OF TWO

(Rev. 11/2005) Page 2-55


Purpose

This program will generate an unduplicated set of random numbers (maximum of 2,000 sets) forthe user. This program may be of use when pairs of random numbers are the most efficient wayto locate a sample item and the numbering of the universe of items either has gaps or thenumbering system repeats within the universe. For example, the universe of transactions that isof interest to the user may be grouped with other transactions in several computer printouts thathave page and line numbers. The transactions of interest may be on pages numbered from 1through 100 in one volume, pages 1 through 456 in a second volume, and pages 45 through 832in a third volume. This program could be used with the existing numbering system to select pageand line numbers.

Within each frame there may be a consistent number of items, such as 66 lines per page, or theitems could vary from frame to frame. This program allows the user to work with eithersituation.

Input Screen

FRAMES - SETS OF TWO RAT-STATS User’s Guide

Page 2-56 (Rev. 11/2005)






The quantity of random numbers to be generated in sequential order should be entered in thisbox. After the quantity indicated has been generated by the program, the random numbers willbe sorted and the output will be in ascending order to assist the user in retrieving the sampleitems. The order of selection will be printed with the random numbers. If the quantity desired iszero, then this box can be left blank or a “0” (zero) can be entered.



Enter the number of frames in your universe (Max = 51)

A frame is a grouping of contiguous sample items within the universe. In the illustration tofollow, the user would have three frames. The first frame would contain pages 1 through 100 inthe first volume, the second frame would contain pages 1 through 456 in the second volume, andthe third frame would contain pages 45 through 832 in the third volume.

Enter or View low/high values

Based on the frame count entered by the user, the user must enter the low and high boundariesfor each frame. When the user clicks on the “Enter or view low/high values” button, the


(Rev. 11/2005) Page 2-57

following form appears. In this illustration, the number of frames is entered as “3” and the“Yes” button is selected in response to the question “Is the range the same within each frame?”. In general, there will be one line in this table for each frame in the universe.

When the “Yes” button is selected in response to the question “Is the range the same within eachframe?”, the low numbers in the second frame set will be set equal to the low number typed inthe first row. For the above illustration, as soon as the user types “1” as the low number for thesecond frame set in the first frame, the value of “1” will appear in this column for frames 2 and 3as soon as the user leaves this cell. Similarly, when “66” is typed as the high number for thesecond frame set in the first frame, the value of “66” will appear in this column for frames 2 and3 as soon as the user leaves this cell.

If the “No” button is selected in response to the question “Is the range the same within eachframe?”, the user must supply all values (e.g., 6 for this illustration) for the low and highnumbers in the second frame set.

Output Options



Page 2-58 (Rev. 11/2005)


Program Output



When all the desired output formats have been selected in the OUTPUT TO box, click on thebutton labeled File Name(s). The user will see the standard Windows “Save” screen. Fill in thename of the file in the File name box. A text file will be saved with a “.TXT” extension (e.g.,C:\TEMP\OUTDISK.TXT). By clicking on the Save button, the program will return to theoriginal input screen for this module.

The output shown below is file C:\ TEMP\OUTDISK.TXT and is the result of generating threerandom numbers along with two spares using the low/high values in the earlier screen with afixed range of 66 in each frame.

For each random number generated, five pieces of information are printed. The first is the orderof selection. This number indicates the order in which the random number was generated by theprogram. The next piece of information is the random number generated. The next three piecesof information are a result of converting the random number into the appropriate frame,subframe, and the item within the subframe. In the sequential portion of the output, the lowestrandom number and the corresponding conversion are printed first. The subsequent numbers arein sequential order with the random number and its equivalent value after conversion on eachline.


(Rev. 11/2005) Page 2-59




FILE OF RANDOM NUMBERS: C:\TEMP\OUTDISK.txt


THE NUMBERS ARE IN THE FOLLOWING FORMAT IN YOUR FILE: POSITIONS 1 THROUGH 6 - ORDER OF SELECTION POSITIONS 7 THROUGH 20 - RANDOM NUMBER POSITIONS 21 THROUGH 29 - FRAME NUMBER POSITIONS 30 THROUGH 42 - SUB-FRAME NUMBER POSITIONS 43 THROUGH 55 - ITEM NUMBER WITHIN THE SUB-FRAMEEACH COLUMN OF NUMBERS IS RIGHT JUSTIFIED.

Selection Random Frame Sub-Frame Item Order Number Number Number Number 2 7559 2 15 35 μ Three random values start here. 3 30214 2 358 52 1 48495 3 223 51 4 33903 2 414 45 μ The two spare values start here. 5 40411 3 101 19


When all the desired output formats have been selected in the OUTPUT TO box, click on thebutton labeled File Name(s). If the OUTPUT TO Access File is selected, the standard Windows“Save” file screen will appear. Fill in the name of the Access database in the File name box. The output file will be saved with the “.MDB” extension (e.g., C:\TEMP\OUTACCESS.MDB). By clicking on the Save button, the program will return to the original input screen for thismodule. After clicking on Continue, the user will see the following form:


Page 2-60 (Rev. 11/2005)

The program is asking for the name of the Access table to create in the specified database(C:\TEMP\OUTACCESS for this illustration). The table name “OutFrame2” will be used. Theprogram will then create two tables (OutFrame2-Values and OutFrame2-Spares) within databaseC:\TEMP\OUTACCESS.

The tables OutFrame2-Values and OutFrame2-Spares, shown next, are the result of generatingthree random numbers along with two spares using the low/high values in the earlier screen witha fixed range of 66 in each frame.

For each random number generated, five pieces of information are printed. The first is the orderof selection. This number indicates the order in which the random number was generated by theprogram. The next piece of information is the random number generated. The next three piecesof information are a result of converting the random number into the appropriate frame,subframe, and the item within the subframe. The seed number used by the random numbergenerator is in the far-right column. In the sequential portion of the output, the lowest randomnumber and the corresponding conversion are printed first. The subsequent numbers are insequential order with the random number and its equivalent value after conversion on each line.

The OutFrame2-Values table contains three random numbers, the seed number, the date, and thetime.


(Rev. 11/2005) Page 2-61


The OutFrame2-Spares table contains two spare values.



The following output will be contained in the Excel spreadsheet C:\TEMP\OUTEXCEL.XLSand is the result of generating three random numbers along with two spares using the low/highvalues in the earlier screen with a fixed range of 66 in each frame. By clicking on the tab labeled“SPARES” the user will obtain the spreadsheet containing the spare values.

For each random number generated, five pieces of information are printed. The first is the orderof selection. This number indicates the order in which the random number was generated by theprogram. The next piece of information is the random number generated. The next three piecesof information are a result of converting the random number into the appropriate frame,subframe, and the item within the subframe. The output also contains the program executiondate and time, the name of the audit/review, the seed number, and the total size of the samplingframe. In the sequential portion of the output, the lowest random number and the correspondingconversion are printed first. The subsequent numbers are in sequential order with the randomnumber and its equivalent value after conversion on each line.

These are the three random values in the Excel spreadsheet named “VALUES” located in C:\TEMP\OUTEXCEL.XLS.


Page 2-62 (Rev. 11/2005)


These are the two spares in the Excel spreadsheet named “SPARES.”



The output immediately following is the text file created by selecting “Flat File” as one of theoutput options. If the OUTPUT TO Flat File is selected, the standard Windows “Save” filescreen will appear. Fill in the name of the file in the File name box. The name of the file forthis illustration is C:\TEMP\OUTFLAT.

This flat file is the output file when generating three random numbers along with two sparesusing the low/high values from the earlier screen with a fixed range of 66 in each frame. Noticethat the order of selection and the random values contain leading zero values. This file is oftenuseful as an input file for selecting random records using a mainframe computer.

For each random number generated, five pieces of information are printed. The first is the orderof selection. This number indicates the order in which the random number was generated by theprogram. The next piece of information is the random number generated. The next three piecesof information are a result of converting the random number into the appropriate frame,


(Rev. 11/2005) Page 2-63


subframe, and the item within the subframe. In the sequential portion of the output, the lowestrandom number and the corresponding conversion are printed first. The subsequent numbers arein sequential order with the random number and its equivalent value after conversion on eachline.





00020000007559000200000000150000000035 μ The three random values start here.000300000302140002000000035800000000520001000004849500030000000223000000005100040000033903000200000004140000000045 μ The two spare values start here.00050000040411000300000001010000000019

Output to a Printer

If the user selects the printer for output, the standard Windows “Print” dialog box will appear. Select the appropriate printer and click on OK.

The following printer output is the result of generating three random numbers along with twospares using the low/high values in the earlier screen with a fixed range of 66 in each frame. Foreach random number generated, five pieces of information are printed. The first is the order ofselection. This number indicates the order in which the random number was generated by theprogram. The next piece of information is the random number generated. The next three piecesof information are a result of converting the random number into the appropriate frame,subframe, and the item within the subframe. In the sequential portion of the output, the lowestrandom number and the corresponding conversion are printed first. The subsequent numbers arein sequential order with the random number and its equivalent value after conversion on eachline.


Page 2-64 (Rev. 11/2005)





ORDER OF RANDOM - - - CONVERTED NUMBER - - - - SELECTION NUMBER FRAME SUB-FRAME ITEM NUMBER

2 - 7,559 2 15 353 - 30,214 2 358 521 - 48,495 3 223 51

====================NEW PAGE===================





ORDER OF RANDOM - - - CONVERTED NUMBER - - - - SELECTION NUMBER FRAME SUB-FRAME ITEM NUMBER

4 - 33,903 2 414 455 - 40,411 3 101 19



(Rev. 11/2005) Page 2-65


Output to Screen


RHC SAMPLE SELECTION RAT-STATS User’s Guide

Page 2-66 (Rev. 11/2005)


Purpose

This procedure will select primary units for a two-stage design and both primary and secondaryunits for a three-stage design using the Rao, Hartley, and Cochran (RHC) methodology. Usingthis approach, all primary (or secondary if it is a three-stage sample) units are randomlydistributed initially into groups. The number of groups is equal to the number of primary (orsecondary) units to be sampled. Within each group, one primary (or secondary) unit is sampledbased upon probability proportional to size.

The program requires that a data file of all primary (or secondary) units be created. The universein the data file may not exceed 6,000 units.

The program will identify sub-units for sampling but will not identify the final sample items forreview. Another random number program must be used for the final sample selection. Forexample, the user may want to review Medicare claims at hospitals throughout the United States. The program may first be used to randomly select states. The program may then be used againto randomly select hospitals within the selected states. Another sampling program, such assingle stage numbers, could then be used to select the Medicare claims for review. The userdoes not need to know the size of the universe of hospitals for each state when performing thefirst phase (i.e., state sampling) of the RHC sample selection. For the second phase, the usermust know and create a data file showing information for each hospital in the states sampled. However, the user does not need to know the number of Medicare claims at each hospital inorder to perform this phase of the RHC sample selection. Once the sampled hospitals have beenidentified, the user must determine the quantity of Medicare claims at each hospital selected inorder to draw a random sample of claims.

While the program may be used for either the primary or secondary unit selection, for ease ofdiscussion in this section, only the primary will be mentioned.

RAT-STATS User’s Guide RHC SAMPLE SELECTION

(Rev. 11/2005) Page 2-67


Input Screen for Selecting Primary/Secondary Units

Selecting the Number of Primary/Secondary Units to be Sampled

Enter the number of units to be sampled in the “How many units?” box. The quantity must notbe greater than the universe of units. The program will randomly assign the units to groups. Thenumber of groups will be equal to the sample size entered.


Page 2-68 (Rev. 11/2005)

Seed Values

The program allows a seed number to be entered by the user when generating the groups ofunits. This is called seed #1 in the above screen. This is the first phase of the sample selection. If no number is entered, the program will use an algorithm to generate its own seed number. Theseed number algorithm is based on the clock in the computer. If the computer clock is notfunctioning, the user should obtain and document a seed number from another source (such as abook of random numbers). The seed number will be displayed as part of the output. If “Willspecify a seed value for partitioning the secondary units (seed #1)” is selected, a box will appearon the screen for entering this value. This option allows the user to obtain the same groups ofprimary units in subsequent computer runs if the same seed value is supplied in this box.

The program allows a seed number to be entered by the user when generating the sample unitfrom each group of units generated in the first phase. This is called seed #2 in the above screen. If no number is entered, the program will use an algorithm to generate its own seed number. Theseed number algorithm is based on the clock in the computer. If the computer clock is notfunctioning, the user should obtain and document a seed number from another source (such as abook of random numbers). The seed number will be displayed as part of the output. If “Willspecify a seed value for selecting the secondary units (seed #2)” is selected, a box will appear onthe screen for entering this value. This option allows the user to obtain the same sample unitsfrom each group of units generated during the first phase in subsequent computer runs if thesame seed value is supplied in this box.

Input File Information

To specify information on the input file, click on the button below:

The following form will appear:


(Rev. 11/2005) Page 2-69

Specify the input file format as (1) Text File (any file extension is accepted), (2) Access database(file extension must be .MDB), or (3) Excel spreadsheet (file extension must be .XLS). Next,click on Open Input File, select the input file, and click on Open.

Format of Input File

The user must create a data file of information on each unit in the universe. If this was anapplication to generate a sample of secondary units, then information would only be needed onthe secondary units of the sampled primary units.

The data for each unit will consist of a description of the unit (e.g., Jefferson County), thequantity of sub-units within the unit (e.g., hospitals in Jefferson County), and a size value for theunit (e.g., number of hospital beds in Jefferson County). The user should try to use a sizeindicator that most closely relates to the purpose of the sample. For example, while the numberof hospital beds in a county may be the best size value for a sample of hospitals, the squarefootage of hospital space may be the best size indicator in another sample. The size value usedaffects the variance in the appraisal. However, the sample is not invalidated by using a less-than-best size indicator. The same measure of size must be used throughout the data file. Theuser, for example, may not use the number of hospital beds for one county and the hospitalsquare footage for another county.


Page 2-70 (Rev. 11/2005)

Regardless of the software used to create the data file, the format should be as follows:

NEW YORK 416 18

Explanation:

NEW YORK - This is a description of a unit in the universe. It may be a number or aname related to the unit. There is a maximum limit of 30 characters forthe description. Commas, hyphens, and other special characters may beused.

416 - This is the quantity of subunits within a given unit. If the user wanted tosample payroll records at branch offices, this value could represent thenumber of payroll records at a particular branch office. If the quantity isnot known, the user may enter a 1 as the universe size. Commas may beused in entering the numbers.

18 - This is the size value associated with a particular unit. For example, if theuser was sampling payroll records at branch offices, this value mayrepresent the number of employees at a particular office. Commas anddecimal points may be used in entering the value.

Input from a Text File

If the input file is stored in a text file and the user selects “Text File” in the Input File Formatlist, the standard Windows “Open” file screen will appear. Click on the file name(INTEXT.TXT for this illustration) and click on Open (or simply double-click on the file name).

COUNTY ONE 1 2990COUNTY TWO 1 576COUNTY THREE 1 2086COUNTY FOUR 1 127COUNTY FIVE 1 682COUNTY SIX 1 307 Input file INTEXT.TXTCOUNTY SEVEN 1 1226COUNTY EIGHT 1 301COUNTY NINE 1 21342COUNTY TEN 1 166COUNTY ELEVEN 1 252COUNTY TWELVE 1 179COUNTY THIRTEEN 1 493COUNTY FOURTEEN 1 93COUNTY FIFTEEN 1 209


(Rev. 11/2005) Page 2-71

COUNTY SIXTEEN 1 163COUNTY SEVENTEEN 1 1871COUNTY EIGHTEEN 1 2008COUNTY NINETEEN 1 253COUNTY TWENTY 1 300COUNTY TWENTY-ONE 1 2754COUNTY TWENTY-TWO 1 2869COUNTY TWENTY-THREE 1 5202COUNTY TWENTY-FOUR 1 2557COUNTY TWENTY-FIVE 1 1770COUNTY TWENTY-SIX 1 358COUNTY TWENTY-SEVEN 1 651COUNTY TWENTY-EIGHT 1 455COUNTY TWENTY-NINE 1 1772COUNTY THIRTY 1 250COUNTY THIRTY-ONE 1 135COUNTY THIRTY-TWO 1 662COUNTY THIRTY-THREE 1 344COUNTY THIRTY-FOUR 1 175COUNTY THIRTY-FIVE 1 927COUNTY THIRTY-SIX 1 350COUNTY THIRTY-SEVEN 1 600COUNTY THIRTY-EIGHT 1 260COUNTY THIRTY-NINE 1 502COUNTY FORTY 1 182

Input From an Access Database

If the input file is stored in a table within an Access database and the user selects “Access File (.MDB)” in the Input File Format list, the standard Windows “Open” file screen will appear,listing all Access database (.MDB) files. Click on the database file name(C:\TEMP\VARIABLE.MDB for this illustration) and click on Open (or simply double-click onthe database file name VARIABLE). Next, the user will be asked to select the name of the tablewithin the selected database using the following form. Click on the down arrow under Select atable.


Page 2-72 (Rev. 11/2005)

After selecting a table from the drop-down list and clicking on Click here to see field names,the following form will appear:

To select the field names, click on the field name for the field containing the Unit Description(Field1 in this illustration) and click on the top right-hand box. The field name will appear in


(Rev. 11/2005) Page 2-73

this box. Repeat this for the field name containing the Number of Sub-Units and click on themiddle right-hand box to specify this field name (Field2 in this illustration). Repeat using thefield name containing the Unit Size and specify this field (Field3 in this illustration) by clickingon the bottom right-hand box. When the field names have been specified, click on OK. Theprogram will return to the original screen for this module.

Input From an Excel Spreadsheet

If the input file is stored in an Excel spreadsheet and the user selects “Excel File (.XLS)” in theInput File Format list, the standard Windows “Open” file screen will appear, listing all Excel(.XLS) files. Click on the Excel file name (C:\TEMP\INEXCEL.XLS for this illustration) andclick on Open (or simply double-click on INEXCEL.XLS). To return to the original screen forthis module, click on OK.

Output Information

To specify program output information, click on the button below:

The following form will appear:


Page 2-74 (Rev. 11/2005)

Format for Complete Output

This output contains information on the groups of primary/secondary units that were formed andthe primary/secondary unit selected from each group. This information can be output to a textfile, a printer, or screen. If either “Text File” or “Printer” is selected, the user will be providedvery brief summary information in the output screen.

If complete information is desired, the user should select either “Text File” or “Printer.” If “TextFile” is selected, the standard Windows “Save” file screen will appear. Enter the File Name(C:\TEMP\OUTRHC.TXT for this illustration) and click on Save. This file will be saved with a“.TXT” extension. If the user selects the printer for output, the standard Windows “Print” dialogbox will appear. Select the appropriate printer and click on OK. The program always concludeswith a brief summary on the screen.

Format for Summary Output

This file is one of the files needed by the RHC appraisal program which is run after the sampleresults have been obtained. This file can be in any one or more of the following formats: (1) aText File, (2) a table within an Access database, or (3) an Excel spreadsheet. When all thedesired output formats have been selected in the Summary Output To box, click on the buttonlabeled File Name(s).


(Rev. 11/2005) Page 2-75

Text Summary File

If output to Text File is selected, the standard Windows “Save” file screen will appear. Type inthe File Name (C:\TEMP\OUTSUMMARY.TXT for this illustration) and click on Save. Thisfile will be saved with a “.TXT” extension. After clicking on Save, the program returns to theoriginal screen for this module. To continue processing the data, click on CONTINUE.

Access Summary File

If the output to Access File is selected, the standard Windows “Save” screen will appear, listingall Access database (.MDB) files. If the database already exists, click on the database name(C:\TEMP\VARIABLE.MDB for this illustration) and click on Save (or simply double click ondatabase VARIABLE). If the database does not exist, type in the database name in the FileName box and click on Save. After clicking on Save, the program returns to the original screenfor this module. To continue processing the data, click on CONTINUE. Next, the user will seethe following message asking for the name of the output table within the previously specifiedAccess database. For this illustration, the Access table name is OutSummary.

Excel Summary File

If the output to Excel File is selected, the standard Windows “Save” file screen will appear,listing all Excel (.XLS) files. If the spreadsheet already exists and the user wants to replace it,click on the spreadsheet name (C:\TEMP\OUTSUMMARY.XLS for this illustration) and thenclick on Save (or simply double-click on the file OUTSUMMARY.XLS). If the spreadsheetdoes not exist, enter the spreadsheet name in the File Name box and click on Save. Afterclicking on Save, the program returns to the original screen for this module. To continueprocessing the data, click on CONTINUE.


Page 2-76 (Rev. 11/2005)

Important NoticeVersions of Windows RAT-STATS prior to RAT-STATS 2005 used executable DOSmodules when generating random numbers. Consequently, the random numbersgenerated agreed with those obtained using DOS versions of RAT-STATS. RAT-STATS 2005 does not utilize these DOS modules but instead uses the full32-bit technology of Visual Basic. As a result, random numbers generated usingRAT-STATS 2005 and later (which includes RAT-STATS 2007) will not agree withthose obtained using any of the previous versions.

Program Output

The Summary Output to file contains information needed by the RHC appraisal program whichis run after the sample results have been obtained. This file can be in any one or more of thefollowing formats: (1) a Text File, (2) a table within an Access database, or (3) an Excelspreadsheet. The following information is contained in the file:

PRIMARY UNIT ID This is the description entered by the user for the unit that wasselected as a sample item.

SECONDARYUNIVERSE

The number of secondary units entered in the data file for the unitbeing sampled.

PRIMARYUNIT SIZE

The size factor entered in the data file for the unit being sampled.

GROUP SIZE The sum of the size factors for the units placed in a particulargroup. The individual and group size will be used by the RHCappraisal program.

UNITS IN GROUP The number of units that were randomly placed into the group.

If the user had selected a secondary unit sample, PRIMARY would be replaced bySECONDARY and SECONDARY would be replaced by 3RD STAGE in all the headingsshown above.


(Rev. 11/2005) Page 2-77

Complete Output to Text File or Printer

The output created by the “Text File” option and the “Printer” option is identical and shownbelow for the sample illustration. This is file C:\TEMP\OUTSUMMARY.TXT specified earlier. Refer to the preceding Program Output section for the column descriptions in this output.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/14/2004 GENERATION OF PRIMARY UNIT SAMPLE Time: 10:35 NAME OF INPUT FILE: C:\TEMP\INTEXT.TXT


********* GROUP 1 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= COUNTY THIRTY-NINE 502 1 COUNTY TWENTY-EIGHT 455 1 COUNTY TWO 576 1 COUNTY TWENTY-NINE 1,772 1 COUNTY SIX 307 1


********* GROUP 2 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= COUNTY THIRTY-SIX 350 1 COUNTY FORTY 182 1 COUNTY ONE 2,990 1 COUNTY THIRTY-TWO 662 1 COUNTY SEVEN 1,226 1


********* GROUP 3 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= COUNTY FOUR 127 1 COUNTY TWENTY-TWO 2,869 1 COUNTY FIVE 682 1 COUNTY THIRTY 250 1 COUNTY THIRTY-ONE 135 1


********* GROUP 4 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= COUNTY SIXTEEN 163 1


Page 2-78 (Rev. 11/2005)

COUNTY TWENTY-FOUR 2,557 1 COUNTY NINETEEN 253 1 COUNTY TWENTY-THREE 5,202 1 COUNTY THIRTY-THREE 344 1


********* GROUP 5 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= COUNTY FOURTEEN 93 1 COUNTY THIRTY-EIGHT 260 1 COUNTY TWENTY-SEVEN 651 1 COUNTY FIFTEEN 209 1 COUNTY THIRTY-FOUR 175 1


********* GROUP 6 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= COUNTY EIGHTEEN 2,008 1 COUNTY TWENTY-SIX 358 1 COUNTY TEN 166 1 COUNTY SEVENTEEN 1,871 1 COUNTY THIRTY-SEVEN 600 1


********* GROUP 7 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= COUNTY ELEVEN 252 1 COUNTY TWELVE 179 1 COUNTY EIGHT 301 1 COUNTY THREE 2,086 1 COUNTY NINE 21,342 1


********* GROUP 8 ********* PRIMARY UNIT SECONDARY PRIMARY UNIT IDENTIFICATION SIZE UNIVERSE ============================== ============= ============= COUNTY TWENTY 300 1 COUNTY TWENTY-ONE 2,754 1 COUNTY THIRTY-FIVE 927 1 COUNTY TWENTY-FIVE 1,770 1 COUNTY THIRTEEN 493 1



(Rev. 11/2005) Page 2-79


NOTE: Example is for illustrative purposes only. The sample size may notconform to the organization’s minimum sample size standards.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/14/2004 GENERATION OF PRIMARY UNIT SAMPLE Time: 10:35 NAME OF OUTPUT FILE: C:\TEMP\OutSummary.txt



UNITS SECONDARY PRIMARY INPRIMARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP========================= ============= ============= ============= =====COUNTY TWENTY-NINE 1 1,772 3,612 5COUNTY ONE 1 2,990 5,410 5COUNTY TWENTY-TWO 1 2,869 4,063 5COUNTY TWENTY-THREE 1 5,202 8,519 5COUNTY TWENTY-SEVEN 1 651 1,388 5COUNTY SEVENTEEN 1 1,871 5,003 5COUNTY NINE 1 21,342 24,160 5COUNTY TWENTY-ONE 1 2,754 6,244 5

The last eight lines in the output are the same information contained in the file specified in theSummary Output To list. This is one of the files required in the RHC appraisal program.

Summary Output to Text File

The file C:\TEMP\OUTSUMMARY.TXT follows. This is identical to the last eight lines of thecomplete output file.

COUNTY TWENTY-NINE 1 1772 3612 5COUNTY ONE 1 2990 5410 5COUNTY TWENTY-TWO 1 2869 4063 5COUNTY TWENTY-THREE 1 5202 8519 5COUNTY TWENTY-SEVEN 1 651 1388 5COUNTY SEVENTEEN 1 1871 5003 5COUNTY NINE 1 21342 24160 5COUNTY TWENTY-ONE 1 2754 6244 5


Page 2-80 (Rev. 11/2005)



Summary Output to Access Database

The table OutSummary is shown below and contains the same information contained in the lasteight lines of the complete output file.

Summary Output to Excel Spreadsheet

The Excel spreadsheet named C:\TEMP\OUTSUMMARY.XLS is shown below and contains thesame information contained in the last eight lines of the complete output file.



(Rev. 11/2005) Page 2-81


Summary Output to Screen


This screen contains the two seed values and the number of sampled primary/secondary units. The contents of the large box consist of the same information contained in the files specified inthe Program Output list.

Attribute Appraisals

RAT-STATS User’s Guide ATTRIBUTE APPRAISAL OVERVIEW

(Rev. 11/2005) Page 3-1

UNRESTRICTED

STRATIFIED



RHC TWO STAGE

RHC THREE STAGE

STRATIFIED CLUSTER

OVERVIEW

The purpose of an attribute sample is to determine the number of items that meet a given set ofcriteria. Typically, in such a sample methodology, the reviewer will conclude after analyzing asample item that the item does or does not meet the stated criteria. The criteria may be as simpleas whether an approving signature is on a document or as complicated as to whether a tax returnmet all of the IRS rules and regulations. The conclusion, however, is usually stated as yes or no.

While the conclusion may be quite simple, the methods used to select the sample may requirequite sophisticated appraisal techniques. The reviewer, for example, may want to group statesby certain characteristics and then sample from each group of states and still be able to make anoverall statement about a given set of criteria.

This package offers the user appraisal methodologies used in designing and performing astatistical sample. A brief example for using each program is given below. A detailedexplanation of how to use each module is included later in this section.

ATTRIBUTE APPRAISAL OVERVIEW RAT-STATS User’s Guide

Page 3-2 (Rev. 11/2005)

Unrestricted

This module is typically used when an unrestricted sample has been drawn. A reviewer mayhave analyzed a sample of timecards from a pay period to determine the number of timecardsthat show employees arriving late for work. An unrestricted random sample of timecards wasused to select the cards for review.

Stratified

In certain cases the reviewer may want to divide the overall universe of transactions into two ormore categories (strata). However, the reviewer may still want to be able to make a statisticalstatement about the overall universe. Expanding on the timecard review mentioned above, thereviewer may want to place each timecard into one of three categories: (1) clerical workers,(2) professional staff, or (3) management. A separate sample would be drawn from each group(stratum). This program will develop statistics for each stratum as well as providing overallstatistics.


The cost of performing a review may cause the reviewer to use a sampling method that couldhelp reduce the costs. For example, the timecards mentioned above could be located throughoutthe United States. The reviewer could randomly select locations throughout the organization andthen, at the selected locations, select a sample of timecards. This sample design could reducetravel costs for the reviewer.


This methodology is similar to the “Two-Stage Unrestricted” with the addition of another levelof sampling. Using the timecard example, the reviewer may decide to select regions of thecountry, then sample locations within the selected regions. Then at the selected locations, theuser would select timecards.

RHC Two Stage

In certain situations a user may want to draw a multistage sample with the probability greater forselecting “larger” units in the universe. For example, a user may want to take an inventory ofitems at various warehouses. However, the user may want the larger warehouses to have agreater chance for selection. The RHC statistical methodology allows the user to weight theprimary units (e.g., by using square footage at each warehouse) and thereby increase the chancefor larger units to be sampled. It provides a method of sample selection that allows samplingwithout replacement while resembling the use of probability proportional to size sampling.

RAT-STATS User’s Guide ATTRIBUTE APPRAISAL OVERVIEW

(Rev. 11/2005) Page 3-3

When the primary units are selected, the size of each primary unit is considered rather thanobtaining a simple random sample of primary units.

The primary units are selected using the RHC Sample Selection program. A random sample isthen obtained for each selected primary unit and the number of secondary units having theattribute of interest (e.g., in error) is recorded.

RHC Three Stage

This is similar to the two-stage RHC procedure in that the primary units are selected byconsidering the size of each unit. In addition, the size of the secondary units is considered whenselecting them from the sampled primary units.

The primary and secondary units are selected using the RHC Sample Selection program. Arandom sample of third-stage units is then obtained for each sampled primary/secondary unitcombination and the number of third-stage units having the attribute of interest (e.g., in error) isrecorded.

Stratified Cluster

The selection of a sample item may sometimes be extremely costly in terms of time andresources. However, once the item has been selected it can be reviewed rather quickly. Forexample, using the timecards again, once the reviewer has arrived at the sampled location, it maytake only a short amount of additional time to review all timecards as compared to a sample ofthe cards. The reviewer may decide to group the locations by number of employees, with largelocations having more than 100 employees. From each group (stratum) the reviewer wouldsample locations. At each selected location, the reviewer would analyze all the timecards.


This methodology is similar to the “Stratified Cluster” with the exception that not all items in thesubuniverse are reviewed. In the example discussed above, the assumption was made that thereviewer had sufficient time and resources to analyze all of the timecards at the selectedlocations. This may not be possible. Therefore, this methodology is used when a sample is stillneeded at each selected location.

These appraisal programs provide the correct statistical results only when the proper sampledesign has been executed. If an unrestricted sample of timecards were drawn from throughoutthe organization, only the unrestricted attribute appraisal program would generate the properresults. Therefore, the reviewer must be sure at the time the sample approach is developed thatthe appropriate appraisal methodology will be used.

ATTRIBUTE - UNRESTRICTED RAT-STATS User’s Guide

Page 3-4 (Rev. 11/2005)


UNRESTRICTED

Purpose

This program performs an attribute appraisal on data input by the user based on an unrestrictedrandom sample. Attribute sampling is used to determine how frequently an event or type oftransaction occurs in a given universe. This type of sampling usually requires a yes or no (trueor false) evaluation of each sampling unit by the user. The results are usually reported as apercentage.

Input Screen

RAT-STATS User’s Guide ATTRIBUTE - UNRESTRICTED

(Rev. 11/2005) Page 3-5

Name of audit/review

This program allows the user to enter a brief description of the audit or purpose of theevaluation. The description, which appears at the top of the output, is limited to 40 characters inlength and may include commas and spaces.

Universe size

The universe size is the total number of items from which the sampled items were selected. Thenumber should be entered without commas but, upon exiting this box, the commas will beinserted. The maximum allowable universe size is 2,147,483,647.

Sample size

The sample size is the quantity of items selected by the user from the universe. This number alsoshould be entered without commas.

Number of sample items with characteristic of interest

The user must establish evaluation criteria for all of the sample items. These criteria must beapplied consistently to all items. The user needs to identify all sample items that have met theevaluation criteria (“characteristic of interest”). Depending on the purpose of the appraisal, theuser would enter the number of items meeting or failing to meet the criteria. For example, if theuser was looking at 100 documents to see if the documents had the proper approval signature,then the characteristic of interest would be the approval signature. If the evaluation of thesample showed that 88 documents out of 100 had the proper approval, the user would enter theresponse to the number of sample items with characteristic of interest as 88. The user could alsoenter 12 (i.e., 100 - 88) if the purpose of the appraisal was to estimate the percentage of theuniverse of documents did not have the approval signature. The program will also evaluatesamples that have:

1. Zero occurrences of the evaluation criteria2. All sample items meeting the evaluation criteria.

Output Options

The program allows for three types of output. The user may select the output to be sent to a textfile, printer, or screen. The user selects the appropriate output by clicking the correspondingbutton.


Page 3-6 (Rev. 11/2005)

The output options are a text file and screen; a printer and screen: a text file, printer, and screen,or screen only. The program always concludes with a summary on the screen.

If the user selects a text file for output, the standard Windows “Save” file screen will appear. Fill in the name of the file in the File name box. A text file will be saved with a “.TXT”extension (e.g., C:\TEMP\OUTDISK.TXT). By clicking on the Save button, the program willreturn to the original input screen for this module.

If the user selects the printer for output, the standard Windows “Print” dialog box will appear. By selecting the appropriate printer and clicking on OK, the program will return to the originalinput screen for this module.

To continue the appraisal, click on CONTINUE.

Program Output

The program will reprint the data supplied by the user (universe size, sample size and number ofsample items with the characteristic of interest) and also provide the appraisal results. Theprecision information is given at the two-sided 80%, 90%, and 95% confidence levels. Thefollowing information will also be displayed:

PROJECTEDQUANTITY INUNIVERSE

The proportion of sample items with the characteristic of interestmultiplied by the universe size.

PERCENT The proportion of sample items with the characteristic of interestdisplayed as a percentage.

STANDARDERROR

An estimate of the standard deviation of the point estimate for theproportion of sample items and the universe total having thecharacteristic of interest. This is a measure of the sample precision.


(Rev. 11/2005) Page 3-7

CONFIDENCELEVELS

The confidence (80%, 90%, or 95%) that the actual proportion (or totalnumber in the universe) will fall within the corresponding confidenceinterval.

LOWER LIMIT The lower boundary of the confidence interval. The limit is shown asboth a number and percentage of the universe. The confidence levelsare 80%, 90%, and 95%.

UPPER LIMIT The upper boundary of the confidence interval. The limit is shown asboth a number and percentage of the universe. The confidence levelsare 80%, 90%, and 95%.

In the event the sample contains zero items having the characteristic of interest, the user will seethe following screen:

If the user responds with “Yes,” the program will only compute the upper limit and the lowerlimit will not be computed. If the user responds with “No,” the program will compute both thelower and upper limits.

In the event the number of sample items with the characteristic of interest is the same as thesample size, the user will also see the preceding screen. If the user responds with “Yes,” theprogram will only compute the lower limit and the upper limit will not be computed. If the userresponds with “No,” the program will compute both the lower and upper limits.

Output to a Text File or Printer

Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTDISK.TXT, shown next. The printer output is identical.


Page 3-8 (Rev. 11/2005)


Department of Health and Human Services OIG - Office of Audit ServicesDate: 10/22/2004 Single Stage Attribute Appraisal Time: 10:48 AUDIT/REVIEW: Attribute SRS


CONFIDENCE LIMITS




In the event that (1) the sample contained no items with the characteristic of interest or (2) thenumber of sample items with the characteristic of interest is equal to the sample size, the user hasthe option of selecting either a one-sided confidence interval (only one of the limits isdetermined) or the usual two-sided interval (both limits are determined). For example, thefollowing output will be obtained for the situation where no items of interest were found in asample of size 400 and the user answered “Yes” to the query “Would you like to compute a one-sided confidence interval? If you answer YES, a one-sided interval will be computed. If youanswer NO, the usual two-sided confidence interval will be computed.”


(Rev. 11/2005) Page 3-9


Department of Health and Human Services OIG - Office of Audit ServicesDate: 10/22/2004 Single Stage Attribute Appraisal Time: 10:52 AUDIT/REVIEW: Attribute SRS

UNIVERSE SIZE 10,000 SAMPLE SIZE 400 CHARACTERISTIC(S) OF INTEREST QUANTITY IDENTIFIED IN SAMPLE 0 PROJECTED QUANTITY IN UNIVERSE 0 PERCENT .000% STANDARD ERROR PROJECTED QUANTITY 0 PERCENT .000%

CONFIDENCE LIMITS

80% CONFIDENCE LEVEL UPPER LIMIT - QUANTITY 39 PERCENT .390%



SINCE NO ITEMS HAVING THE CHARACTERISTIC(S) OF INTEREST WERE FOUND IN THESAMPLE, THE PROGRAM HAS CALCULATED ONLY THE MAXIMUM NUMBER OF ITEMSHAVING THE CHARACTERISTIC(S) OF INTEREST IN THE UNIVERSE.

If the preceding sample had contained 400 items having the characteristic of interest and the userspecified a one-sided interval, the above output would only contain the lower limits, along withthe message “SINCE ALL SAMPLE ITEMS CONTAINED THE CHARACTERISTIC(S) OFINTEREST, THE PROGRAM HAS CALCULATED ONLY THE MINIMUM NUMBER OFITEMS IN THE UNIVERSE HAVING THE CHARACTERISTIC(S) OF INTEREST.”


Page 3-10 (Rev. 11/2005)


Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedis text file or printer. The following screen is the summary provided for this illustration:

The following screen will be obtained for the situation where no items of interest were found in asample of size 400 and the user answered “Yes” to the query “Would you like to compute a one-


(Rev. 11/2005) Page 3-11


sided confidence interval? If you answer YES, a one-sided interval will be computed. If youanswer NO, the usual two-sided confidence interval will be computed.”

If the preceding sample had contained 400 items having the characteristic of interest and the userspecified a one-sided interval, the preceding screen would only contain the lower limits, alongwith the message “Since all sample items had the characteristic(s) of interest, the program hascalculated only the minimum number of items having the characteristic(s) of interest in theuniverse for each confidence level.”

ATTRIBUTE - STRATIFIED RAT-STATS User’s Guide

Page 3-12 (Rev. 11/2005)


STRATIFIED

Purpose

This program provides a stratified attribute appraisal from the user’s input to a series of prompts. Attribute sampling is used to determine how frequently an event or type of transaction occurs ina given universe. This type of sampling usually requires a yes or no (true or false) evaluation ofeach sampling unit by the user. The results are usually reported as a percentage.

Input Screen

RAT-STATS User’s Guide ATTRIBUTE - STRATIFIED

(Rev. 11/2005) Page 3-13

NOTE: Example is for illustrative purposes only. The sample sizes maynot conform to the organization’s minimum sample size standards.



Number of strata

After entering the name of the audit/review, the user must enter the number of strata. Afterentering the number of strata, click on OK. The following screen will appear:


Page 3-14 (Rev. 11/2005)

Have you created a data file?

The user may create a data file prior to running this appraisal. For each stratum, the user needsto enter the universe size, sample size, and number of items with the characteristic of interest. The program also allows the user to edit the values and save the modified file. The values mayalso be entered from the keyboard and subsequently saved as a data file. If the user has notcreated a data file, select “No” in the above box. If “Yes” is selected, the standard Windows“Open” file screen will appear. The window will contain all “.TXT” and “.DAT” files in theselected directory. To view all files in this directory, the user may change the file type to “Allfiles.” Click on the file name, then click on Open. If a file name is entered, the values obtainedfrom the file will be displayed on the screen for review by the user. The format for the datashould be as follows:

1000 100 2

Explanation:

1000 - This is the size of the universe of items from which a sample was drawn in astratum.

100 - This is the quantity of sample items that were reviewed.

2 - This is the quantity of items that met the criteria established for the samplereview.

The program assumes a comma, one or more spaces, or a tab as a delimiter between pieces ofdata. The user needs to enter large numbers (e.g., 10000) without commas (e.g., 10,000) sincethe program will assume that the comma is separating two pieces of data (e.g., 10 and 000). Theuser must not use dollar signs ($) or any other symbols in conjunction with the data as theprogram will assign a value of zero to any input that contains nonnumeric data.

Entering the Sample Results From the Screen

The sample results can be entered from the screen form if a data file was not used to enter theresults. For this input option, the user will be presented a grid, as shown next. To illustrate,suppose stratum 1 contains 1,000 values, 100 of which are sampled, and 2 of the items containthe characteristic of interest. Also, stratum 2 contains 1,500 values, 100 of which are sampled,and 6 of the items contain the characteristic of interest. NOTE: This grid structure does not


(Rev. 11/2005) Page 3-15


allow the user to use the tab key to move from cell to cell. Each cell must be clicked on beforeentering its value.

When all the values within the grid have been entered, click on Click here when finishedentering or editing your data, the Save Input Data button will appear.

To save this data set, click on Save Input Data. The standard Windows “Save As” screen willappear. Type the output file name alongside the File name box and click on Save. To print thisdata set, click on Print Input Data. Select a printer and click on OK.


Page 3-16 (Rev. 11/2005)

Output Options

The program allows for three types of output. The user may select the output be sent to text file,printer or screen. The user selects the appropriate output by clicking the corresponding button.

The output options are a text file and screen; a printer and screen; a text file, printer, and screen,or screen only. The program always concludes with a summary on the screen.


If the user selects the printer for output, the standard Windows “Print” dialog box will appear. Select the appropriate printer and click on OK and the program will return to the original inputscreen for this module.


Program Output

The program will output the data supplied by the user (universe size, number of items withcharacteristic of interest, and sample size) for each stratum. The precision information is givenat the two-sided 80%, 90%, and 95% confidence levels. The following information will also bedisplayed:

RATIO The percentage of sample items in each stratum with the characteristicof interest.

PROJ. ITEMSIN UNIVERSE

The result of applying the stratum ratio to the stratum universe. This isalso calculated for the total universe.


(Rev. 11/2005) Page 3-17

STANDARDERROR

A measurement of the standard deviation of the sample proportion withrespect to all possible proportions for this universe and sample size.

PRECISION The confidence interval half-width expressed as a percentage. Precisionis calculated for each stratum and the universe as a whole.

CONFIDENCELEVEL (CL)


LOWER LIMIT The lower boundary of the confidence interval. The limit is shown asboth a number and a percentage of the universe. The confidence levelsare 80%, 90%, and 95%.

UPPER LIMIT The upper boundary of the confidence interval. The limit is shown asboth a number and a percentage of the universe. The confidence levelsare 80%, 90%, and 95%.


Page 3-18 (Rev. 11/2005)



Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTDISK.TXT, shown next. The printer output is identical but does not include theoutput file name if the output was not also saved to a text file. Both the text file and printeroutput will include the name of the input file if the data were input in this manner.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 STRATIFIED ATTRIBUTE APPRAISAL Time: 10:37 AUDIT/REVIEW: Attribute - Stratified OUTPUT FILE: C:\TEMP\OUTDISK.TXT

PROJ. ITEMSSTRATUM SAMPLE *ITEMS** **RATIO* *UNIVERSE* IN UNIVERSE======= ====== ======== ======== ========== ===========

1 100 2 2.000% 1,000 20 2 100 6 6.000% 1,500 90

COMBINED 200 8 4.400% 2,500 110

STANDARD ERROR: 1.483% 37

PRECISION PRECISION PRECISIONSTRATUM AT 80% CL AT 90% CL AT 95% CL======= ========= ========= ========= 1 1.711% 2.196% 2.616% 2 2.955% 3.793% 4.519%

COMBINED 1.901% 2.439% 2.907%

LOWER LIMIT - QUANTITY 62 49 37 PERCENT 2.499% 1.961% 1.493%UPPER LIMIT - QUANTITY 158 171 183 PERCENT 6.301% 6.839% 7.307%


(Rev. 11/2005) Page 3-19


Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedis a text file or printer. The following screen is the summary provided for this illustration. Thissummary will also include the name of the input file if the data were entered in this manner.

ATTRIBUTE - TWO-STAGE UNRESTRICTED RAT-STATS User’s Guide

Page 3-20 (Rev. 11/2005)



Purpose

This program provides a two-stage attribute appraisal from the user’s input to a series ofprompts. Attribute sampling is used to determine how frequently an event or type of transactionoccurs in a given universe. This type of sampling usually requires a yes or no (true or false)evaluation of each sampling unit by the user. The results are usually reported as a percentage. The program will accept a maximum of 150 primary sampling units for an appraisal.

Input Screen

RAT-STATS User’s Guide ATTRIBUTE - TWO-STAGE UNRESTRICTED

(Rev. 11/2005) Page 3-21



Number of primary units in the universe:

This is the number of primary units in the universe from which the primary items were sampled. For example, the primary universe could be counties within a state. From a universe of 58counties, the user might sample 12 counties. The response to this query would be 58.

Number of primary units in the sample:

Enter the number of sampled primary units in this box. For example, from a universe of 58counties, the user sampled 12 counties. The response to this query would be 12.

Number of secondary units in the universe (if known):

This is the total number of all transactions or events (secondary units) in the universe. In theabove example, secondary units could be claims in each county. The response to this querywould be the claim count for all 58 counties. This total number of secondary items may bedifficult to determine. Therefore, the user may enter a zero if the quantity is unknown.

If a zero is entered, a different set of formulas will be used that utilize a ratio-type estimator. This estimator is statistically biased, but the bias is negligible if the number of sampled primaryunits is large.

If the total number of secondary units is entered, the formulas use an unbiased estimator.

When all values have been specified, click on OK. The following screen will appear:


Page 3-22 (Rev. 11/2005)


The user may create a data file prior to running this appraisal. For each primary unit, the userneeds to enter the universe size, sample size, and number of items with the characteristic ofinterest. The program also allows the user to edit the values and save the modified file. Thevalues may also be entered from the keyboard and subsequently saved as a data file. If the userhas not created a data file, select “No” in the above box. If “Yes” is selected, the standardWindows “Open” file screen will appear. Click on the file name (DATA2STG.TXT for thisillustration) and click on Open (or simply double-click on file DATA2STG.TXT).


(Rev. 11/2005) Page 3-23


If a file name is entered, the values obtained from the file will be displayed on the screen forreview by the user. The format for the data should be as follows:

50 10 4

Explanation:

50 - This is the number of items in the primary unit from which a sample was drawn.

10 - This is the quantity of sample items that were reviewed.

4 - This is the quantity of items that met the criteria established for the samplereview.

The program assumes a comma, one or more spaces, or a tab as a delimiter between pieces ofdata. The user needs to enter large numbers (e.g., 10000) without commas (e.g., 10,000) sincethe program will assume that each comma is separating two pieces of data (e.g., 10 and 000). The user must not use dollar signs ($) or any other symbols in conjunction with the data as theprogram will assign a value of zero to any value that contains nonnumeric data.

The text file C:\TEMP\DATA2STG.TXT used in this illustration is shown below. Ten primaryunits are sampled from a total of 90. The number of secondary units in the universe is known tobe 4,500.

50 10 445 9 252 10 542 8 340 8 265 13 548 10 358 12 366 13 456 11 4


Page 3-24 (Rev. 11/2005)

Entering the Sample Results from the Screen

The sample results can be entered from the screen form if a data file was not used to enter theresults. To illustrate, suppose the first primary unit contains 50 universe values, 10 of which aresampled. Also, the number of sample items meeting the criteria (e.g., in error) is 4. Thesevalues should be entered as shown on the screen. NOTE: This grid structure will not allow theuser to use the tab key to move from cell to cell. Each cell must be clicked on before entering itsvalue.

When all the values within the grid have been entered, click on Click here when finishedentering or editing your data and the Save Input Data button will appear.


(Rev. 11/2005) Page 3-25


Output Options

The program allows for three types of output. The user may select the output be sent to text file,printer, or screen. The user selects the appropriate output by clicking the corresponding button.



If the user selects the printer for output, the standard Windows “Print” dialog box will appear. Select the appropriate printer and click on OK and the program will return to the original inputscreen for this module.


Program Output

The program will print the data supplied by the user (universe size, sample size, and number ofsample items with characteristic of interest) for each primary unit. Overall totals are shown forthe number of primary units in the universe and the number of secondary items in the universe (if


Page 3-26 (Rev. 11/2005)

known). The precision information is given at the two-sided 80%, 90%, and 95% confidencelevels. The following information will also be displayed for each primary unit:

RATIO The percentage of sample items in each primary unit with thecharacteristic of interest.

PROJECTED If the total number of secondary items in the universe was entered, thenthe projected number of items meeting the characteristic(s) of interestfor each primary unit will be displayed.

The following overall statistics will also be displayed.

OVERALLRATIO

The estimate of the universe proportion, stated as a percentage.

OVERALLPROJECTED

The proportion of sample items with the characteristic of interestmultiplied by the number of secondary items in the universe (if known).

STANDARDERROR

A measurement of the standard deviation of the sample proportion withrespect to all possible proportions for this universe and sample size.

CONFIDENCELEVELS


LOWER LIMIT The lower limit of the 80%, 90%, and 95% confidence interval. Thelimit is shown as both a proportion of the universe and quantity of itemsif the user entered a non-zero value for the universe of secondary items.

UPPER LIMIT The upper limit of the 80%, 90%, and 95% confidence interval. Thelimit is shown as both a proportion of the universe and quantity of itemsif the user entered a non-zero value for the universe of secondary items.


(Rev. 11/2005) Page 3-27



Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTDISK.TXT, shown next. The printer output is identical but does not include theoutput file name if the output was not also saved to a text file. Both the text file and printeroutput will include the name of the input file if the data were input in this manner.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 TWO STAGE UNRESTRICTED ATTRIBUTE APPRAISAL Time: 10:56 AUDIT/REVIEW: Attribute - 2 Stage DATA FILE: C:\TEMP\DATA2STG.TXT OUTPUT FILE: C:\TEMP\OUTDISK.TXT


1 50 10 4 40.00% 20 2 45 9 2 22.22% 10 3 52 10 5 50.00% 26 4 42 8 3 37.50% 16 5 40 8 2 25.00% 10 6 65 13 5 38.46% 25 7 48 10 3 30.00% 14 8 58 12 3 25.00% 15 9 66 13 4 30.77% 20 10 56 11 4 36.36% 20

TOTALS 522 104 35

OVERALL TOTALS 90 4,500 35.26% 1,587





Page 3-28 (Rev. 11/2005)


Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedis a text file or printer. The following screen is the summary provided for this illustration:

RAT-STATS User’s Guide ATTRIBUTE - THREE-STAGE UNRESTRICTED

(Rev. 11/2005) Page 3-29


Purpose

This program computes the point estimate, standard error, and confidence limits for the universepercentage and universe total number when using a three-stage attribute sampling procedure. Attribute sampling is used to determine how frequently an event or type of transaction occurs ina given universe. The results can be reported as a percentage estimate or an estimate of the totalnumber in the universe having this characteristic of interest. The program will accept amaximum of 50 first-stage units and 150 second-stage units within each first-stage unit (i.e., 50 x150 = 7,500 second-stage units).

Input Screen

ATTRIBUTE - THREE-STAGE UNRESTRICTED RAT-STATS User’s Guide

Page 3-30 (Rev. 11/2005)



Number of primary units in the universe:

This is the number of primary units in the universe from which the primary items were sampled. For example, the primary universe could be regions within the United States. From a universe of12 regions, one might sample four regions. The response to this query would be 12.

Number of third-stage units in the universe (if known):

If the number of third-stage units in the universe is known, enter this value. If this universe sizeis known, an unbiased estimate of the universe proportion is obtained. If this universe size isunknown, enter a value of zero, in which case a ratio-type estimator of the universe proportion isused by the program. This estimator is biased, but the bias is negligible if the number ofsampled primary units is large.

Read input file

When all values have been specified, click on the Read Input File button.

For this program, the user must enter the data from a file (i.e., screen input is not allowed). Afterclicking on Read Input File, the standard Windows “Open” file screen will appear. Click on thefile name (DATA3ST.TXT for this illustration) and click on Open (or simply double-click onfile DATA3ST.TXT).

The data file containing the information must be stored in a text file format. There are severalways the user may create this file. The easiest approach would be with a word processingpackage (e.g., Word) or storing the data in a spreadsheet software application (e.g., Excel) as aprint file. The data file needs to be formatted in the following manner:


(Rev. 11/2005) Page 3-31

REGION 5 90 10HOSPITAL 1 47 9 3

Explanation:

REGION 5 - is the description of a primary unit. The description may be of any length upto 25 characters and include characters, digits, spaces, and punctuation marks.

90 - is the universe of secondary units within the primary unit.

10 - is the number of secondary units that are being sampled in the primary unit.

HOSPITAL 1 - is a description of a secondary unit that was sampled from the primary unit. The description may be of any length up to 25 characters and includecharacters, digits, spaces, and punctuation marks.

47 - is the number of third-stage items comprising the universe for the secondarysampled unit.

9 - is the number of third-stage items that were sampled.

3 - is the number of third-stage sampled items that met the characteristic ofinterest to the user.

The last piece of data (e.g., 10) on each primary unit line is used by the program to determine thenumber of lines with secondary unit information. The program, therefore, expects that numberof lines (e.g., 10) to be lines with secondary unit information. Each line of data must containinformation on only one primary unit or one secondary unit. Values within a line can beseparated by commas, one or more spaces, or tabs. Commas should not be used within numericvalues (e.g., use 2500 rather than 2,500).

The data file used in this illustration is stored in C:\TEMP\DATA3ST.TXT and is shown below. The primary units are regions within the U.S. and the secondary units are hospitals within eachregion.

REGION 5 90 10HOSPITAL 1 47 9 3HOSPITAL 2 51 10 2HOSPITAL 3 45 9 4HOSPITAL 4 46 9 1HOSPITAL 5 46 9 3HOSPITAL 6 50 10 1HOSPITAL 7 50 10 4


Page 3-32 (Rev. 11/2005)


HOSPITAL 8 57 11 3HOSPITAL 9 54 11 4HOSPITAL 10 64 13 2REGION 7 110 10HOSPITAL 1 53 11 2HOSPITAL 2 59 12 5HOSPITAL 3 52 10 1HOSPITAL 4 67 13 3HOSPITAL 5 59 12 1HOSPITAL 6 73 15 6HOSPITAL 7 51 10 3HOSPITAL 8 75 15 2HOSPITAL 9 66 13 1HOSPITAL 10 58 12 4REGION 8 85 10HOSPITAL 1 45 9 3HOSPITAL 2 39 8 2HOSPITAL 3 43 9 4HOSPITAL 4 34 7 1HOSPITAL 5 54 11 2HOSPITAL 6 54 11 3HOSPITAL 7 34 7 1HOSPITAL 8 59 12 1HOSPITAL 9 49 10 4HOSPITAL 10 43 9 2REGION 10 120 10HOSPITAL 1 59 12 2HOSPITAL 2 68 14 6HOSPITAL 3 57 11 3HOSPITAL 4 72 14 6HOSPITAL 5 70 14 1HOSPITAL 6 73 15 2HOSPITAL 7 83 17 5HOSPITAL 8 89 18 4HOSPITAL 9 73 15 3HOSPITAL 10 77 15 2


(Rev. 11/2005) Page 3-33

After opening the data set, the remaining portion of the input screen will appear.

The primary unit data (e.g., Primary Unit Number, Primary Unit ID, Number of secondary stageunits in universe and Secondary stage units in sample) cannot be changed on this screen. Formore information, click on the Info button. The user may scroll through the secondary unit dataor edit any of those cells by clicking on the cell to be changed. NOTE: This grid structure willnot allow the user to use the tab key to move from cell to cell. Each cell must be clicked onbefore entering its value.

If any values within the grid are edited, click on Click here when finished editing your data,the Save Input Data button will appear.


Page 3-34 (Rev. 11/2005)


Output Options




If the user selects the printer for output, the standard Windows “Print” dialog box will appear. Select the appropriate printer and click on OK, and the program will return to the original inputscreen for this module.


Program Output

The first section of the program output contains the name of the data file along with the contentsof this file. For each secondary unit in the file, the value of RATIO is printed, where RATIO isthe percentage of third-stage sample units within this secondary unit having the characteristic ofinterest. Totals are shown for each primary unit summarizing (1) the number of third-stage unitsin the universe, (2) the number of sampled third-stage units, and (3) the number of third-stageunits having the characteristic of interest.


(Rev. 11/2005) Page 3-35

The next section of output, entitled OVERALL TOTALS, summarizes the above three values(across all primary units in the sample) along with the number of primary and secondary units inthe universe and sample. The universes, with respect to the second and third stages, reflect thepopulations for stages reviewed and not the universes across all stages. For example if 10primary units were sampled from a population of 100 primary units, the universe of secondaryunits displayed in the output represents only the quantity of secondary units in the 10 primaryunits. The precision information is given at the two-sided 80%, 90%, and 95% confidencelevels. The following information is also presented:

OVERALL POINTESTIMATE OF THEPROPORTION

This is the estimate of the universe proportion having thecharacteristic of interest.

OVERALLSTANDARD ERROR(PROPORTION)

An estimate of the standard deviation of the proportion estimate;a value which determines the width of the correspondingconfidence intervals for the parameter.

OVERALL POINTESTIMATE OFUNIVERSE TOTAL

The estimate of the total number of elements in the universehaving the characteristic of interest.

OVERALLSTANDARD ERROR(TOTAL)

An estimate of the standard deviation of the universe totalestimate; a value which determines the width of thecorresponding confidence intervals for the parameter.

CONFIDENCE LEVELS The confidence (80%, 90%, or 95%) that the actual proportion(or total number in the universe) will fall within thecorresponding confidence interval.

LOWER LIMITFOR PROPORTION

The lower limit of the 80%, 90%, and 95% confidence intervalfor the universe proportion.

UPPER LIMITFOR PROPORTION

The upper limit of the 80%, 90%, and 95% confidence intervalfor the universe proportion.

LOWER LIMITFOR TOTAL

The lower limit of the 80%, 90%, and 95% confidence intervalfor the universe total number.

UPPER LIMITFOR TOTAL

The upper limit of the 80%, 90%, and 95% confidence intervalfor the universe total number.


Page 3-36 (Rev. 11/2005)


Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTDISK.TXT, shown immediately following. The printer output is identical butdoes not include the output file name if the output was not also saved to a text file.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 THREE STAGE ATTRIBUTE APPRAISAL Time: 11:13 AUDIT/REVIEW: Attribute - 3 Stage NAME OF INPUT FILE: C:\TEMP\DATA3ST.txt OUTPUT FILE: C:\TEMP\OUTDISK.TXT

FIRST STAGE NEXT STAGE MEETING SECOND STAGE UNIVERSE SAMPLE SIZE CRITERIA RATIO============================ ============ =========== ======== =======REGION 5 90 10 HOSPITAL 1 47 9 3 33.33% HOSPITAL 2 51 10 2 20.00% HOSPITAL 3 45 9 4 44.44% HOSPITAL 4 46 9 1 11.11% HOSPITAL 5 46 9 3 33.33% HOSPITAL 6 50 10 1 10.00% HOSPITAL 7 50 10 4 40.00% HOSPITAL 8 57 11 3 27.27% HOSPITAL 9 54 11 4 36.36% HOSPITAL 10 64 13 2 15.38%

TOTALS 510 101 27

REGION 7 110 10 HOSPITAL 1 53 11 2 18.18% HOSPITAL 2 59 12 5 41.67% HOSPITAL 3 52 10 1 10.00% HOSPITAL 4 67 13 3 23.08% HOSPITAL 5 59 12 1 8.33% HOSPITAL 6 73 15 6 40.00% HOSPITAL 7 51 10 3 30.00% HOSPITAL 8 75 15 2 13.33% HOSPITAL 9 66 13 1 7.69% HOSPITAL 10 58 12 4 33.33%

TOTALS 613 123 28


(Rev. 11/2005) Page 3-37


TOTALS 454 93 23


TOTALS 721 145 34

TOTAL NUMBER OF THIRD STAGE UNITS IN UNIVERSE 72,500

OVERALL TOTALS UNIVERSE SAMPLED======= ====== ======== ======= FIRST STAGE 12 4 SECOND STAGE 405{} 40 THIRD STAGE 2,298{} 462 SAMPLED ITEMS MEETING CRITERIA 112 {} UNIVERSE SIZES FOR THE SECOND AND THIRD STAGES REPRESENT THE UNIVERSES FOR THE SAMPLED PRIOR STAGE.

OVERALL POINT ESTIMATE OF THE PROPORTION 23.74%OVERALL STANDARD ERROR (PROPORTION) 3.33%

OVERALL POINT ESTIMATE OF UNIVERSE TOTAL 17,210OVERALL STANDARD ERROR (TOTAL) 2,415




Page 3-38 (Rev. 11/2005)


Output to Screen


RAT-STATS User’s Guide ATTRIBUTE - RHC TWO STAGE

(Rev. 11/2005) Page 3-39

RHC TWO STAGE

Purpose

This program performs a two-stage attribute appraisal using the RHC methodology. Thismultistage procedure determines a confidence interval that uses approximate probabilityproportional to size (pps) sampling whereby the relative sizes of the sampling units areconsidered when selecting primary units to include in the sample.

Input Screen

ATTRIBUTE - RHC TWO STAGE RAT-STATS User’s Guide

Page 3-40 (Rev. 11/2005)



Prior to executing this program, the user must create a data file that contains identifying data andtwo pieces of information for each secondary unit sampled. The first piece of information is thesample size for this secondary unit. The second value in this line is the number of sample itemsthat met the criteria established for the sample review. These two pieces of information shouldbe separated by one or more spaces, commas, or tabs. Commas should not be used withinnumeric values (e.g., use 2500 rather than 2,500).

The data file containing the above information must be stored in a text file format. There areseveral ways the users may create this file. The easiest approach would be with a wordprocessing package (e.g., Word) or a print file created with a spreadsheet package (e.g., Excel).

7483 200 42

Explanation:

7483 - This is a number assigned by the user. The user should use the sample itemnumber as the number in this position. For ease of reference in this example,the number will be referred to as the line number.

200 - This is the sample size (number of examined units) for this secondary unit.

42 - This is the number of sample items that met the criteria established for thesample review.

The file used in this illustration will be C:\TEMP\RHC2DATA.TXT. After clicking on OpenData File, locate this file using the standard Windows “Open” file screen and click on Open.

When sample items are selected using the RHC Sample Selection program, a file is createdcontaining information on the clustering of the primary units in the universe as well as the


(Rev. 11/2005) Page 3-41

primary units selected. The Open Primary Unit File query is requesting the file created fromthe RHC Sample Selection program (e.g., C:\TEMP\RHC2PU.TXT).

If the user cannot find this file, a file containing the following data must be created before theappraisal can be performed. The file must be stored in a text file format. The are several waysthe user may create this file. The easiest approach would be with a word processing package(e.g., Word) or a print file created with a spreadsheet package (e.g., Excel).

PRIMARY #1 1500 20100 95500 6

Explanation:

PRIMARY #1 - This is a description of a primary unit in the universe. A maximum of 30characters, including spaces, will be accepted.

1500 - This is the quantity of secondary units in the particular primary unit.

20100 - This is the size factor assigned by the user to the primary unit (e.g., number oftransactions or square footage). A decimal point may be included in thisvalue.

95500 - This is the total size factor for the group in which the primary unit wasrandomly placed. A decimal point may be included in this value.

6 - This is the quantity of primary units that were randomly placed into thisgroup.

The above entries for each primary unit must be placed on the same line of text. Each line oftext must only include the entries for one primary unit. Values within a line can be separated bycommas, by one or more spaces, or by using the tab key. Commas should not be used withinnumeric values (e.g., use 2500 rather than 2,500).

After clicking on Open Primary Unit File, locate this file using the standard Windows “Open”file screen and click on Open. The screen immediately following the data files will appear.

The data file used in this illustration contains data for 27 universities having one or more state-supported research grants. The universe consists of all charge vouchers recorded for these grantsand the purpose of the audit is to estimate the proportion of vouchers containing impropercharges. Three universities (UNIV5, UNIV20, and UNIV22) were selected using the RHCSample Selection program. For each selected university, 250 vouchers were audited. Thesample data file and primary unit file are shown next.


Page 3-42 (Rev. 11/2005)

Sample data file C:\TEMP\RHC2DATA.TXT

1 250 8 2 250 12 3 250 5

Primary unit file C:\TEMP\RHC2PU.TXT

UNIV5 15989 11 67 9UNIV20 15400 10 93 9UNIV22 17522 13 93 9

The user cannot edit any of the values in the grid. To print this data set, click on Print InputData. Select a printer and click on OK.



(Rev. 11/2005) Page 3-43

Output Options

The program allows for three types of output. The user may select the output to be sent to textfile, printer, or screen. The user selects the appropriate output by clicking the correspondingbutton.





Program Output

The output will first display the data entered by the user for each of the primary units. Then theoutput will display the results of the appraisal. The following will be displayed in the resultsportion.

P.U. NBR For conserving space, the sample primary unit from each group isassigned the group number. For example, the primary unit sampledfrom group #1 is assigned the number 1 for identification purposes.


Page 3-44 (Rev. 11/2005)

PRIMARYUNIT ID

The description assigned by the user to each primary unit.

SECONDARYUNIVERSE

The number of secondary units in the population of a particular primaryunit.

PRIMARY UNIT SIZE

The size value selected by the user for assigning a weight to eachprimary unit. The weight factor could be, for example, number ofemployees, volume of transactions, or square footage.

GROUP SIZE The summation of the size values for all the primary units in theparticular group.

UNITSIN GROUP

The number of primary units that were placed randomly in a particulargroup.

For each sampled primary unit, the following information is displayed.

SAMPLE SIZE The number of sampled secondary units contained in this particularsampled primary unit.

SAMPLE MEAN The proportion of secondary units containing the characteristic ofinterest. It is obtained by dividing the number of items in the samplecontaining the characteristic of interest by the number of sample items.

SECONDARYUNIVERSE

The number of secondary units in the universe for this particularsampled primary unit. This value was originally supplied by the user inthe file containing the Primary Unit information.

SIZES RATIO The ratio of the size of the group containing this particular primary unitto the size of the primary unit itself.

POINTESTIMATE

The estimate of the universe total for the group of primary unitscontaining this particular primary unit. For example, suppose theprimary units are split into 10 random groups, each containing 5primary units. Suppose further that the primary unit under discussionlies in group #8. The POINT ESTIMATE refers to the estimate of theuniverse total of the five primary units in group #8. This would berepeated for the remaining sampled primary units. The point estimatesare then totaled to obtain the estimate of the universe total.


(Rev. 11/2005) Page 3-45

--- VARIANCE COMPONENTS —

WITHINVARIANCE

In the derivation of the standard error, the contribution of the variabilityof the secondary units.

BETWEENVARIANCE

In the derivation of the standard error, the contribution of the variabilityof the primary units.

TOTALVARIANCE

The sum of WITHIN VARIANCE and BETWEEN VARIANCE. Thesquare root of this value is the STANDARD ERROR.

The results of the above information are then used in the final overall projection. The followinginformation appears:

PRIMARYUNITSSAMPLED

The number of primary units in the sample.

PRIMARYUNITS NOTSAMPLED

The number of primary units in the population minus the number ofprimary units in the sample.

PRIMARYUNITS INPOPULATION

The total number of primary units in the population.


The point estimate of population total. A single estimate for a universevalue based on the summation of the point estimates for each group.

STANDARDERROR

A measurement of the standard deviation of the estimate for thepopulation total. It is this value that determines the width of thecorresponding confidence intervals.

CONFIDENCELEVEL

This indicates the confidence (80%, 90%, or 95%) that the user has thatthe actual population total will fall within the corresponding confidenceinterval.

LOWER LIMIT The lower bound of the confidence interval. It is based on subtractingthe precision amount from the point estimate.


Page 3-46 (Rev. 11/2005)

UPPER LIMIT The upper bound of the confidence interval. It is based on adding theprecision amount to the point estimate.

PRECISIONAMOUNT

A measurement of the closeness of the sample estimate and thecorresponding population value. For a 90% confidence interval, theuser would be 90% confident that the estimated population total(PROJECTED QUANTITY IN UNIVERSE) would be within thisamount of the actual value. The precision amount is calculated bymultiplying the standard error by the appropriate Z value (Z-VALUEUSED).

PRECISIONPERCENT

The result of dividing the precision amount by the point estimate.

Z-VALUE USED The standard normal percentile value used to construct the confidenceinterval.


(Rev. 11/2005) Page 3-47


Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTDISK.TXT, shown below. The printer output is identical.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 TWO STAGE RHC ATTRIBUTE APPRAISAL Time: 15:11 AUDIT/REVIEW: RHC 2-Stage

NAME OF DATA FILE: C:\TEMP\RHC2DATA.TXT NAME OF PRIMARY UNIT FILE: C:\TEMP\RHC2PU.TXT OUTPUT FILE: C:\TEMP\OUTDISK.txt

PRIMARY SAMPLE == ATTRIBUTE == UNIT SIZE SAMPLE TOTAL ======= ====== ============= 1 250 8 2 250 12 3 250 5

TOTALS 750 25

UNITSP.U. SECONDARY PRIMARY INNBR PRIMARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP==== ========================= ============= ============= ============= ===== 1 UNIV5 15,989 11 67 9 2 UNIV20 15,400 10 93 9 3 UNIV22 17,522 13 93 9

TOTALS: 48,911 34 253 27

P.U. SAMPLE SECONDARY SIZES POINT NBR SIZE SAMPLE MEAN UNIVERSE RATIO ESTIMATE ==== ====== ============== ============= ======== ============= 1 250 .03 15,989 6.091 3,116.40 2 250 .05 15,400 9.300 6,874.56 3 250 .02 17,522 7.154 2,506.99

TOTALS: 750 48,911 12,497.96


P.U. WITHIN BETWEEN TOTAL NBR VARIANCE VARIANCE VARIANCE ==== ======== ======== ======== 1 190,680.61 62,730.47 253,411.08 2 398,194.39 6,287,762.71 6,685,957.11 3 170,420.81 5,266,818.04 5,437,238.85

TOTALS: 759,295.82 11,617,311.23 12,376,607.04



Page 3-48 (Rev. 11/2005)

PROJECTED QUANTITY IN UNIVERSE: 12,498STANDARD ERROR: 3,518




(Rev. 11/2005) Page 3-49

Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedis text file or printer. The following screen is the summary provided for this illustration:


ATTRIBUTE - RHC THREE STAGE RAT-STATS User’s Guide

Page 3-50 (Rev. 11/2005)

RHC THREE STAGE

Purpose

This program performs a three-stage attribute appraisal using the RHC methodology. Thismultistage procedure determines a confidence interval that uses approximate probabilityproportional to size (pps) sampling whereby the relative sizes of the sampling units areconsidered when selecting primary and secondary units to include in the sample.

Input Screen

RAT-STATS User’s Guide ATTRIBUTE - RHC THREE STAGE

(Rev. 11/2005) Page 3-51



Prior to executing this program, the user must create a data file that contains identifying data andtwo pieces of information for each third-stage item sampled. The first piece of information is thesample size for this third-stage unit. The second value in this line is the number of sample itemsthat met the criteria established for the sample review. These two pieces of information shouldbe separated by one or more spaces, commas, or tabs.

The data file containing the above information must be stored in a text file format. There areseveral ways the users may create this file. The easiest approach would be with a wordprocessing package (e.g., Word) or a print file created with a spreadsheet package (e.g., Excel).

4.12 180 10

Explanation:

4.12 - This identifies the primary unit number (4, here) and the secondary unitnumber (12, here). The primary unit numbers should be numberedsequentially (1, 2, 3, ...) and the secondary unit numbers within a sampledprimary unit should also be numbered sequentially.

180 - This is the number of third-stage units sampled within this particular primaryunit/secondary unit.

10 - This is the number of sample items that met the criteria established for thesample review.

The program assumes a comma, one or more spaces, or a tab as a delimiter between pieces ofdata. The user needs to enter large numbers (e.g., 10000) without commas (e.g., 10,000) sincethe program will assume that each comma is separating two pieces of data (e.g., 10 and 000). The user must not use dollar signs ($) or any other symbols in conjunction with the data as theprogram will assign a value of zero to any input that contains nonnumeric data.


Page 3-52 (Rev. 11/2005)

The file used in this illustration will be C:\TEMP\RHC3DATA.TXT. After clicking on OpenData File, locate this file using the standard Windows “Open” file screen and click on Open.

When sample items are selected using the RHC Sample Selection program, a file was createdcontaining information on the clustering of the primary units/secondary units in the universe aswell as the primary units/secondary units selected. The Open P.U./S.U. File query is requestingthe file created from the RHC Sample Selection Program. The primary unit information isentered first, followed by the information for the secondary units. The format is as follows:

PRIMARY #1 20 9000 30600 8 3SECONDARY #1 3500 1000 4400 10

Explanation:

PRIMARY #1 - This is a description of the sampled primary unit. The description shouldbe no longer than 30 characters in length.

20 - This is the quantity of secondary units in this primary unit.

9000 - This is the size factor assigned by the user to the primary unit. A decimalpoint may be included in this value.

30600 - This is the size factor for the group from which the primary unit wasselected. A decimal point may be included in this value.

8 - This is the number of primary units in the group from which this primaryunit was selected.

3 - This is the number of secondary units sampled from this primary unit forthis appraisal.

SECONDARY #1 - This is a description of the sampled secondary unit. A maximum of 30characters, including spaces, will be accepted.

3500 - This is the quantity of third-stage units in this secondary unit. The entrymay contain a decimal point.


(Rev. 11/2005) Page 3-53

1000 - This is the secondary size factor used in weighting the secondary unit. Adecimal point may be incorporated in the number.

4400 - This is the size factor for the secondary group that this secondary item wassampled from. A decimal point may be used in the value.

10 - This is the number of secondary units in this group from which thissecondary unit was selected.

The program assumes a comma, one or more spaces, or a tab as a delimiter between pieces ofdata. The user needs to enter large numbers (e.g., 10000) without commas (e.g., 10,000) sincethe program will assume that each comma is separating two pieces of data (e.g., 10 and 000). The user must not use dollar signs ($) or any other symbols in conjunction with the data as theprogram will assign a value of zero to any input that contains nonnumeric data.

After clicking on Open P.U. / S.U. File, locate this file within the standard Open File windowand click on Open. Next, click on READ FILES and the screen immediately following the datafiles will appear.

The data file used in this illustration contains data for three U.S. regions (primary units) and universities (secondary units) within each region having one or more state-supported researchgrants. The universe consists of all charge vouchers (third-stage units) recorded for these grantsand the purpose of the audit is to estimate the proportion of vouchers containing impropercharges. Four regions (REGION3, REGION7, REGION8, REGION9) were selected from auniverse of 12 using the RHC Sample Selection program. Using the RHC Sample Selectionprogram on each region, 10 universities are selected. For each selected university,approximately 20% of the vouchers were audited. The data file and primary unit/secondary unitfile immediately follow.

Sample Data file C:\TEMP\RHC3DATA.TXT

1.1 7 2 μ Data for Region 31.2 15 41.3 15 31.4 12 21.5 12 51.6 6 21.7 11 21.8 8 41.9 5 11.10 11 32.1 11 2 μ Data for Region 72.2 11 42.3 13 2


Page 3-54 (Rev. 11/2005)

2.4 16 62.5 13 32.6 6 22.7 8 32.8 11 22.9 13 22.10 12 33.1 7 1 μ Data for Region 83.2 15 63.3 10 23.4 8 43.5 14 33.6 10 33.7 5 03.8 5 13.9 13 03.10 15 34.1 13 5 μ Data for Region 94.2 15 64.3 14 14.4 14 34.5 11 34.6 15 44.7 10 04.8 7 24.9 5 24.10 11 5

Primary unit file C:\TEMP\RHC3PUSU.TXT

REGION3 91 720 3280 3 10UNIV20 37 6 73 9UNIV38 74 11 70 9UNIV45 73 11 82 9UNIV10 60 9 81 9UNIV87 62 10 59 9UNIV82 30 5 68 9UNIV60 54 9 75 9UNIV69 39 7 76 9UNIV34 26 4 60 9UNIV54 57 9 76 10REGION7 102 960 2210 3 10UNIV1 56 10 89 10UNIV60 56 10 96 10UNIV59 67 13 94 10UNIV99 80 14 91 10UNIV85 67 13 93 10UNIV37 31 6 103 10UNIV34 42 8 106 10UNIV16 53 10 83 10UNIV12 66 13 90 11UNIV52 60 11 115 11REGION8 118 1300 3710 3 10


(Rev. 11/2005) Page 3-55

UNIV19 34 8 137 11UNIV104 77 16 122 11UNIV66 49 11 122 12UNIV110 38 9 137 12UNIV83 70 15 121 12UNIV14 48 10 147 12UNIV78 27 7 122 12UNIV105 27 6 144 12UNIV12 65 14 122 12UNIV112 75 16 126 12REGION9 122 1320 2800 3 10UNIV37 64 14 147 12UNIV92 73 15 125 12UNIV47 71 15 130 12UNIV54 70 15 131 12UNIV97 56 12 138 12UNIV66 76 16 122 12UNIV116 50 10 140 12UNIV29 33 8 128 12UNIV18 26 7 132 13UNIV107 55 11 127 13



Page 3-56 (Rev. 11/2005)

The user cannot edit any of the values in the grid. To print this data set, click on Print InputData. Select a printer and click on OK.

Output Options

The program allows for three types of output. The user may select the output to be sent to textfile, printer, or screen. The user selects the appropriate output by clicking the correspondingbutton.


(Rev. 11/2005) Page 3-57





Program Output

The output will first display the data entered by the user for each of the primary units. Then theoutput will display the results of the appraisal. The following will be displayed in the resultsportion:

--- POINT ESTIMATES ---

PRIMARY/SECONDARYIDENTIFICATION

The description information obtained from the file of universe dataidentified by the user.

SAMPLE MEAN The proportion of third-stage items having the characteristic ofinterest. It is obtained by dividing the number of items in thesample having the characteristic of interest by the number of itemsin the sample.

SIZES RATIO The ratio of the size of the group containing this particularsecondary unit to the size of the secondary unit itself. The sizefactors are obtained from the universe file indicated by the user.

POINT ESTIMATE The estimate of the universe total for the group of secondary unitscontaining this particular secondary unit. For example, suppose thesecondary units are split into 10 random groups, each containing5 secondary units. Suppose further that the sampled secondary unitlies in group #8. The point estimate refers to the estimate of theuniverse total of the five secondary units in group #8. This would


Page 3-58 (Rev. 11/2005)

be repeated for the remaining sampled secondary units.


WITHINVARIANCE

For each sampled primary unit, the contribution of the third-stagevariation.

BETWEENVARIANCE

For each sampled primary unit, the contribution of the second-stagevariation.

TOTALVARIANCE

For each sampled primary unit, the sum of WITHIN VARIANCE andBETWEEN VARIANCE. This value represents the total variationobtained by applying a two-stage RHC procedure to the sampledprimary unit.


STAGE 1 In the derivation of the standard error, the contribution of the first-stage(primary) units.

STAGES2 AND 3

In the derivation of the standard error, the contribution of the second-stage (secondary) and third-stage units.

TOTALVARIANCE

The sum of the values for STAGE 1 and STAGES 2 AND 3. Thesquare root of this value is the STANDARD ERROR.

SUMMARY SECTION: The results of the above information are then used in the finaloverall projection. The following information appears:

--- SUMMARY OF APPRAISAL RESULTS ---

PRIMARYUNITSSAMPLED

The quantity of primary units selected in this sample.




(Rev. 11/2005) Page 3-59

TOTALPRIMARYUNITS



The overall point estimate is a single estimate for a universe valuebased on each primary unit point estimate multiplied by (A/B) where Ais the size of the group containing the primary unit, B is the size of theprimary unit, and the products are summed over all primary units.

OVERALLSTANDARDERROR


CONFIDENCELEVEL

This indicates the confidence the user has that the actual populationtotal will fall within the corresponding confidence interval.



PRECISIONAMOUNT

A measurement of the closeness of the sample estimate and thecorresponding population value. For a 90% confidence interval, theuser would be 90% confident that the estimated population total(OVERALL POINT ESTIMATE) would be within this amount of theactual value. The precision amount is calculated by multiplying thestandard error by the appropriate factor (“z” value).

PRECISIONPERCENT




Page 3-60 (Rev. 11/2005)


Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTDISK.TXT, shown on the next page. The printer output is identical.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 THREE STAGE RHC ATTRIBUTE APPRAISAL Time: 16:02 AUDIT/REVIEW: RHC 3-Stage

DATA FILE USED: C:\TEMP\RHC3DATA.TXT PRIMARY/SECONDARY UNIVERSE FILE USED: C:\TEMP\RHC3PUSU.TXT OUTPUT FILE: C:\TEMP\OUTDISK.txt

*** ATTRIBUTE *** **** SAMPLED UNITS **** THIRD STAGE SAMPLE NO. WITHPRIMARY / SECONDARY IDENTIFICATION UNIVERSE SIZE ATTRIBUTE================================== =========== ====== ============REGION3 UNIV20 37 7 2 UNIV38 74 15 4 UNIV45 73 15 3 UNIV10 60 12 2 UNIV87 62 12 5 UNIV82 30 6 2 UNIV60 54 11 2 UNIV69 39 8 4 UNIV34 26 5 1 UNIV54 57 11 3REGION7 UNIV1 56 11 2 UNIV60 56 11 4 UNIV59 67 13 2 UNIV99 80 16 6 UNIV85 67 13 3 UNIV37 31 6 2 UNIV34 42 8 3 UNIV16 53 11 2 UNIV12 66 13 2 UNIV52 60 12 3REGION8 UNIV19 34 7 1 UNIV104 77 15 6 UNIV66 49 10 2 UNIV110 38 8 4 UNIV83 70 14 3 UNIV14 48 10 3 UNIV78 27 5 0 UNIV105 27 5 1 UNIV12 65 13 0 UNIV112 75 15 3REGION9 UNIV37 64 13 5 UNIV92 73 15 6 UNIV47 71 14 1 UNIV54 70 14 3 UNIV97 56 11 3 UNIV66 76 15 4


(Rev. 11/2005) Page 3-61

UNIV116 50 10 0 UNIV29 33 7 2 UNIV18 26 5 2 UNIV107 55 11 5

TOTALS 2,440 433 111

--- POINT ESTIMATES --- *** ATTRIBUTE *** **** SAMPLED UNITS **** SIZES POINT PRIMARY / SECONDARY IDENTIFICATION SAMPLE MEAN RATIO ESTIMATE ================================== =========== =========== =============REGION3 UNIV20 0.29 12.1667 129 UNIV38 0.27 6.3636 126 UNIV45 0.20 7.4545 109 UNIV10 0.17 9.0000 90 UNIV87 0.42 5.9000 152 UNIV82 0.33 13.6000 136 UNIV60 0.18 8.3333 82 UNIV69 0.50 10.8571 212 UNIV34 0.20 15.0000 78 UNIV54 0.27 8.4444 131

TOTAL 1,244


TOTAL 1,359


TOTAL 1,296

REGION9 UNIV37 0.38 10.5000 258 UNIV92 0.40 8.3333 243 UNIV47 0.07 8.6667 44 UNIV54 0.21 8.7333 131 UNIV97 0.27 11.5000 176 UNIV66 0.27 7.6250 155 UNIV116 0.00 14.0000 0


Page 3-62 (Rev. 11/2005)

UNIV29 0.29 16.0000 151 UNIV18 0.40 18.8571 196 UNIV107 0.45 11.5455 289

TOTAL 1,643


**** SAMPLED UNITS **** WITHIN BETWEEN TOTALPRIMARY UNIT IDENTIFICATION VARIANCE VARIANCE VARIANCE============================== ============== ============== ==============REGION3 3,898 15,618 19,516REGION7 4,356 22,749 27,105REGION8 3,952 75,526 79,478REGION9 4,749 77,463 82,212


STAGE 1 STAGES 2 AND 3 TOTAL VARIANCE======================= ======================= ======================= 2,580,562 552,511 3,133,073

*** ATTRIBUTE *** --- SUMMARY OF APPRAISAL RESULTS ---PRIMARY UNITS SAMPLED 4PRIMARY UNITS NOT SAMPLED 8TOTAL PRIMARY UNITS 12

PROJECTED QUANTITY IN UNIVERSE 15,981STANDARD ERROR 1,770




(Rev. 11/2005) Page 3-63

Output to Screen



ATTRIBUTE - STRATIFIED CLUSTER RAT-STATS User’s Guide

Page 3-64 (Rev. 11/2005)

STRATIFIED CLUSTER

Purpose

This program computes the point estimate, standard error, and confidence limits for the universepercentage and universe total number when using a stratified cluster attribute samplingprocedure. Attribute sampling is used to determine how frequently an event or type oftransaction occurs in a given universe. The results can be reported as a percentage estimate or anestimate of the total number in the universe having this attribute. The program will accept aminimum of one stratum and a maximum of 100 strata. A sample of no more than 150 primaryunits is allowed within each stratum (i.e., 100 x 150 = 15,000 primary units). This procedure isused when all secondary items in a sampled primary unit are reviewed for sampling purposes.

Input Screen

RAT-STATS User’s Guide ATTRIBUTE - STRATIFIED CLUSTER

(Rev. 11/2005) Page 3-65



Read input file

When the name of review/audit has been specified, click on the Read Input File button.

Prior to executing this program, the user must create a data file that contains the results of thestratified cluster attribute sample. The data file contains information about each of the primaryand secondary units sampled.

The data file containing the information must be stored in a text file format. There are severalways the user may create this file. The easiest approach would be with a word processingpackage (e.g., Word) or storing the data in a spreadsheet software application (e.g., Excel) as aprint file. Values within a line can be separated by commas, by one or more spaces, or by usingthe tab key. Commas should not be used within numeric values (e.g., use 2500 rather than2,500). The data file needs to be formatted in the following manner:

STATE UNIVERSITIES 415 25 2500 UNIV1 8 2

Explanation:

STATE UNIVERSITIES - is the description of a primary unit. The description may be of anylength up to 25 characters and include characters, digits, spaces,and punctuation marks.

415 - is the number of primary units in the universe for this stratum.

25 - is the number of primary units in the sample for this stratum.

2,500 - is the number of secondary units in the universe for this stratum.

UNIV1 - is a description of a primary unit that was sampled. The


Page 3-66 (Rev. 11/2005)

description may be of any length up to 25 characters and includecharacters, digits, spaces, and punctuation marks.

8 - is the universe of secondary items in the sampled primary unit. The number also represents the quantity of secondary items beingsampled.

2 - is the number of secondary items that met the characteristic ofinterest to the user.

The program assumes a comma, one or more spaces, or a tab as a delimiter between pieces ofdata. The user needs to enter large numbers (e.g., 10000) without commas (e.g., 10,000) sincethe program will assume that each comma is separating two pieces of data (e.g., 10 and 000). The user must not use dollar signs ($) or any other symbols in conjunction with the data as theprogram will assign a value of zero to any value that contains nonnumeric data.

The data file used in this illustration is C:\TEMP\DATACLUS.TXT. The universe consists of583 universities with health-related research grants. The two strata consist of state universities(415 universities) and private universities (168 universities). Within each stratum, a single-stagecluster sample was obtained by sampling 25 state universities and 10 private universities. Thetotal number of grants in the universe is 2,500 (state universities) and 1,000 (private universities)for a total of 3,500 grants in the entire universe. Of interest is the proportion of grants containingcharges after the scheduled completion of the grant.

STATE UNIVERSITIES 415 25 2500UNIV1 8 2UNIV2 12 3UNIV3 4 2UNIV4 5 1UNIV5 6 1UNIV6 6 2UNIV7 7 2UNIV8 5 2UNIV9 8 2UNIV10 3 1UNIV11 2 0UNIV12 6 2UNIV13 5 1UNIV14 10 3UNIV15 9 1UNIV16 3 1UNIV17 6 2UNIV18 5 1UNIV19 5 1


(Rev. 11/2005) Page 3-67


UNIV20 4 1UNIV21 6 1UNIV22 8 1UNIV23 7 2UNIV24 3 1UNIV25 8 2PRIVATE UNIVERSITIES 168 10 1000UNIV1 2 1UNIV2 5 2UNIV3 7 2UNIV4 4 2UNIV5 3 1UNIV6 8 3UNIV7 6 2UNIV8 10 4UNIV9 3 1UNIV10 1 1


Page 3-68 (Rev. 11/2005)

After clicking on Read Input File and opening the data file (i.e., C:\TEMP\DATACLUS.TXT),the following screen will appear:

For help in viewing the information for each stratum contained in the four boxes above the grid,click on the Info button. If any values within the grid are edited, click on Click here whenfinished editing your data and the Save Input Data button will appear.



(Rev. 11/2005) Page 3-69

Output Options


The output options are a text file and screen; a printer and screen; a text file, printer, and screen;or screen only. The program always concludes with a summary on the screen.

If the user selects a text file for output, the standard Windows “Save” file screen will appear. Fill in the name of the file in the File name box. A text file will be saved with a “.txt” extension(e.g., C:\TEMP\OUTDISK.TXT). By clicking on the Save button, the program will return to theoriginal input screen for this module.



Program Output

The first section of the program output contains the name of the data file along with the contentsof this file in summary form. In addition, the output displays the percent meeting the criteria andprojected quantity for each stratum.

The next section of output contains overall strata totals in terms of universes, sample sizes andquantity of items with the characteristic of interest. The precision information is given at thetwo-sided 80%, 90%, and 95% confidence levels. The following information is also presented:

OVERALLTOTAL (%)

The estimate of the universe proportion having the characteristic ofinterest.


Page 3-70 (Rev. 11/2005)

OVERALLTOTAL (#)

The estimate, expressed in terms of secondary units, of items havingthe characteristic of interest.

OVERALLSTANDARDERROR (%)

An estimate of the standard deviation of the proportion estimate; avalue which determines the width of the corresponding confidenceintervals for this parameter.

OVERALLSTANDARDERROR (#)

An estimate of the standard deviation of the universe total estimate; avalue which determines the width of the corresponding confidenceintervals for this parameter.

CONFIDENCELEVEL

The confidence (80%, 90%, or 95%) that the actual proportion (ortotal number in the universe) will fall within the correspondingconfidence interval.

LOWERLIMIT FORTOTAL

The lower limit of the 80%, 90%, and 95% confidence interval forthe universe total number.

UPPERLIMIT FORTOTAL

The upper limit of the 80%, 90%, and 95% confidence interval forthe universe total number.

LOWERLIMIT FORPROPORTION

The lower limit of the 80%, 90%, and 95% confidence interval forthe universe proportion.

UPPERLIMIT FORPROPORTION

The upper limit of the 80%, 90%, and 95% confidence interval forthe universe proportion.


(Rev. 11/2005) Page 3-71



DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 STRATIFIED CLUSTER ATTRIBUTE APPRAISAL Time: 14:35 AUDIT/REVIEW: Attribute - Stratified Cluster NAME OF INPUT FILE: C:\TEMP\DATACLUS.TXT OUTPUT FILE: C:\TEMP\OUTDISK.TXT

STRATUM IDENTIFICATION SAMPLE SAMPLE MEETING PROJECTED CLUSTER IDENTIFICATION UNIVERSE SIZE CRITERIA PERCENT QUANTITY========================== =========== ====== ======== ======= =========STATE UNIVERSITIES 415 25 UNIV1 8 8 2 UNIV2 12 12 3 UNIV3 4 4 2 UNIV4 5 5 1 UNIV5 6 6 1 UNIV6 6 6 2 UNIV7 7 7 2 UNIV8 5 5 2 UNIV9 8 8 2 UNIV10 3 3 1 UNIV11 2 2 0 UNIV12 6 6 2 UNIV13 5 5 1 UNIV14 10 10 3 UNIV15 9 9 1 UNIV16 3 3 1 UNIV17 6 6 2 UNIV18 5 5 1 UNIV19 5 5 1 UNIV20 4 4 1 UNIV21 6 6 1 UNIV22 8 8 1 UNIV23 7 7 2 UNIV24 3 3 1 UNIV25 8 8 2

STRATUM TOTALS 2,500 151 38 25.17% 629


Page 3-72 (Rev. 11/2005)


PRIVATE UNIVERSITIES 168 10 UNIV1 2 2 1 UNIV2 5 5 2 UNIV3 7 7 2 UNIV4 4 4 2 UNIV5 3 3 1 UNIV6 8 8 3 UNIV7 6 6 2 UNIV8 10 10 4 UNIV9 3 3 1 UNIV10 1 1 1

STRATUM TOTALS 1,000 49 19 38.78% 388

STRATA TOTALS 583 35CLUSTER UNIT TOTALS 3,500 200 57OVERALL TOTAL 29.05% 1,017OVERALL STANDARD ERROR 1.32% 46




(Rev. 11/2005) Page 3-73


Output to Screen


ATTRIBUTE - STRATIFIED MULTISTAGE RAT-STATS User’s Guide

Page 3-74 (Rev. 11/2005)


Purpose

This program performs a stratified multistage appraisal based on information gathered from priormultistage appraisals. The user would have initially stratified the clusters (e.g., universities) intotwo or more categories (e.g., public and private universities). Within each stratum, the userwould select a multistage sample. The results of the samples would be appraised using amultistage appraisal program. The point estimate and standard error from each of theseappraisals could be placed in a data file or entered interactively by the user.

Input Screen


RAT-STATS User’s Guide ATTRIBUTE - STRATIFIED MULTISTAGE

(Rev. 11/2005) Page 3-75



Number of strata for this appraisal

The user must specify the number of strata that will be appraised. This number must be between2 and 500.


Prior to executing this program the user can create a data file that contains the point estimate andthe standard error for each of the strata. The data file containing the above information must bestored in a text file format. The program also allows the user to edit the values and save themodified file. The values may also be entered from the keyboard and subsequently saved as adata file. If the user has not created a data file, select “No” in the above box. If “Yes” isselected, select the input file and click on Open.

Once the data file has been created and opened, the values from the file will be displayed on thescreen for the user’s review. For the data in each stratum, the user needs to use the followingformat:

16.90 1.99 5600Explanation:

16.90 - This is the overall estimate for one stratum. This value should be expressedas a percentage (e.g., enter 5 for 5%, not .05).

1.99 - This is the standard error for one stratum. This value should be expressed as apercentage (e.g., enter 5 for 5%, not .05).

5600 - This is the number of sample units in the universe for the stratum. This is theuniverse at the most detailed level of the multistage sample. Commas shouldnot be used within this value.


Page 3-76 (Rev. 11/2005)

The results for each stratum should be on a separate line. The program assumes a comma, one ormore spaces, or a tab as a delimiter between pieces of data. The user needs to enter largenumbers (e.g., 10000) without commas (e.g., 10,000) since the program will assume that eachcomma is separating two pieces of data (e.g., 10 and 000). The user must not use dollar signs ($)or any other symbols in conjunction with the data as the program will assign a value of zero toany value that contains nonnumeric data.

The text file C:\TEMP\DATAMULTI.TXT used in this illustration is shown below. There aretwo strata consisting of state universities (Stratum 1) and private universities (Stratum 2).

16.90 1.99 5600 21.84 3.57 3500

After selecting the data file, the following screen will appear:



(Rev. 11/2005) Page 3-77

Entering the Sample Results From the Screen

If “No” was selected in response to the query “Have you created a data file for this appraisal?”then a blank grid will appear and the user may enter the data values (e.g., 6) directly using thescreen. NOTE: This grid structure will not allow the user to use the tab key to move from cellto cell. Each cell must be clicked on before entering its value.

After all values within the grid have been entered, click on the Click here after entering orediting data button.


Output Options

The program allows for three types of output. The user may select the output be sent to text file,printer or screen. The user selects the appropriate output by clicking the corresponding button.



Page 3-78 (Rev. 11/2005)


If the user selects the printer for output, the standard Windows “Print” dialog box will appear. Select the appropriate printer and click on OK, the program will return to the original inputscreen for this module.


Program Output

The output will first display the data entered by the user for each of the strata. Then the outputwill display the results of the appraisal. The precision information is given at the two-sided80%, 90%, and 95% confidence levels. The following will be displayed in the results portion:

ESTIMATEDPERCENTAGE

A single estimate of the universe percentage for all strata of theuniverse value of the characteristic(s) being measured.

ESTIMATEDTOTAL

A single estimate of the universe total for all strata of the universe valueof the characteristic(s) being measured.

STANDARDERROR(PERCENTAGE)

An estimate of the standard deviation of the proportion estimate; a valuewhich determines the width of the corresponding confidence intervalsfor this parameter.

STANDARDERROR(TOTAL)

An estimate of the standard deviation of the universe total estimate; avalue which determines the width of the corresponding confidenceintervals for this parameter.

CONFIDENCELEVEL

This indicates the confidence that the user has that the actual value ofthe measured characteristic will fall within the range from the lower toupper limits (confidence interval). The confidence levels are 80%,90%, and 95%.

LOWERLIMIT FORPROPORTION

The lower bound of the 80%, 90%, or 95% confidence interval for theuniverse proportion.

UPPERLIMIT FORPROPORTION

The upper bound of the 80%, 90%, or 95% confidence interval for theuniverse proportion.


(Rev. 11/2005) Page 3-79


LOWER LIMITFOR TOTAL

The lower bound of the 80%, 90%, or 95% confidence interval for theuniverse total.

UPPER LIMITFOR TOTAL

The upper bound of the 80%, 90%, or 95% confidence interval for theuniverse total.



DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 STRATIFIED MULTISTAGE ATTRIBUTE APPRAISAL Time: 14:46 AUDIT/REVIEW: Attribute-Stratified Multistage DATA FILE: C:\TEMP\DATAMULTI.TXT OUTPUT FILE: C:\TEMP\OUTDISK.TXT

THE ESTIMATORS ARE BASED ON THE FOLLOWING ENTRIES: STRATUM ESTIMATE STANDARD ERROR UNIVERSE SIZE 1 16.90% 1.99% 5,600 2 21.84% 3.57% 3,500

= = = = = = = = = = = = = = = = = RESULTS = = = = = = = = = = = = = = = = = =

ESTIMATED PERCENTAGE: 18.80%ESTIMATED TOTAL: 1,711

STANDARD ERROR (PERCENTAGE): 1.84%STANDARD ERROR (TOTAL): 167




Page 3-80 (Rev. 11/2005)


Output to Screen


Variable Appraisals

RAT-STATS User’s Guide VARIABLE - OVERVIEW

(Rev. 11/2005) Page 4-1

UNRESTRICTED

STRATIFIED



RHC TWO STAGE

RHC THREE STAGE

STRATIFIED CLUSTER


POSTSTRATIFICATION


OVERVIEW

The purpose of a variable sample is to determine a quantitative characteristic or set ofcharacteristics about a population. The reviewer may want to determine the dollar value of aninventory or the amount of duplicate payments made by an organization. These types ofestimates can be made with a variable sample.

These appraisal programs provide the correct statistical results only when the proper sampledesign has been executed. If a stratified sample of inventory cards were drawn from throughoutthe organization, only the stratified variable appraisal program would generate the proper results. Therefore, the reviewer must be sure at the time the sample approach is developed that theappropriate appraisal methodology will be used.

This package offers the user 10 appraisal methodologies when designing and performing avariable statistical sample. A brief example of when to use each module is given below. Adetailed explanation of how to use each module is described later in this section.

VARIABLE - OVERVIEW RAT-STATS User’s Guide

Page 4-2 (Rev. 11/2005)

Unrestricted

This module is used when an unrestricted sample has been drawn. A reviewer may want todetermine the value of an organization’s inventory. The reviewer may have drawn anunrestricted random sample of inventory cards and has analyzed the related inventory todetermine the actual value. Based on this sample, the reviewer could estimate the total inventoryof the organization.

Stratified

The reviewer may wish to spend more resources analyzing certain inventory items as comparedto other items. One approach to accomplish this objective would be to stratify the inventoryitems into two or more categories (strata). One category (stratum) could be for more sensitiveitems (e.g., high-value items). The appraisal can give the reviewer an estimate for each categoryas well as an overall approximation of the inventory.


The cost of performing a review may affect the sampling methodology used. For example, theinventory could be distributed throughout several locations (primary units) in the United States. A reviewer could randomly select locations and then, from the selected locations, sampleinventory cards to be analyzed. This methodology could save travel costs for the reviewer.


This methodology is similar to the “Two-Stage Unrestricted” with the addition of another levelof sampling. Using the inventory example, the reviewer may decide to select by region of thecountry and then sample locations within the selected regions. Then at the selected locations, theuser would select inventory cards.

RHC Two Stage

In certain situations, the reviewer may want to use a multistage sample with greater probabilityof selecting “larger” units in the universe. For example, a reviewer may want to take aninventory of items at various locations. However, the reviewer may want larger locations tohave a greater chance for selection. The Rao, Hartley and Cochran (RHC) methodology allowsthe reviewer to weight (e.g., by using square footage) the locations (primary units). If thismethodology has been used for selecting the sample items, then this module would be used toappraise the sample results.

RAT-STATS User’s Guide VARIABLE - OVERVIEW

(Rev. 11/2005) Page 4-3

RHC Three Stage

The appraisal program is similar to the “RHC Two-Stage” with the addition of another level ofsampling. For the inventory example, the reviewer may sample by region and then samplelocations within the selected regions. The “RHC Sample Selection” must be used to draw thesample.

Stratified Cluster

The selection of a sample item may sometimes be extremely costly in terms of time andresources. However, once the item has been selected it can be reviewed rather quickly. Forexample, using the inventory cards again, once the reviewer has arrived at the sampled location,it may take only a short amount of time to review all of the inventory items at the location. Thereviewer may decide to group the locations by total inventory value (e.g., locations with totalinventory over $10 million). For each group (stratum) the reviewer would sample locations. Foreach selected location, the reviewer would analyze all inventory cards.


This methodology is similar to “Stratified Cluster” with the exception that not all items in thesubuniverse are reviewed. In the example discussed above, the assumption was made that thereviewer had sufficient time and resources to analyze all of the inventory cards at the selectedlocations. This may not be possible. Therefore, this methodology is used when a sample is stillneeded at each selected location.

Poststratification

The reviewer may wish that the sample had been stratified after evaluating the items. Forexample, after performing an unrestricted sample of inventory cards, the reviewer may see thatcertain types of inventory items had a greater chance of discrepancies between the inventory cardbalance and the actual physical inventory. While post stratification is allowed, if each stratum’suniverse size is known, the results are less efficient statistically as compared to a stratifiedsample design.

Unknown Universe Size

The reviewer may have a situation where the size of the universe of items to be sampled is notknown and cannot be readily determined. The reviewer could draw a sample from a frame thatincludes all the desired items. The results from this sample would be used to estimate theuniverse size. A separate sample would be drawn of review items that meet the reviewer's

VARIABLE - OVERVIEW RAT-STATS User’s Guide

Page 4-4 (Rev. 11/2005)

criteria as a sample item.

Assume the reviewer wants to sample invoices that are in boxes. The reviewer has no idea howmany invoices are in all the boxes. The reviewer could first sample boxes and count the numberof invoices in each selected box. An unrestricted variable appraisal would be used to estimatethe universe size. The reviewer would then perform another sample to select individual invoicesfrom all the boxes. Another unrestricted variable appraisal would be performed. The resultsfrom the two samples would be requested by this program to develop the overall estimate.

RAT-STATS User’s Guide VARIABLE - UNRESTRICTED

(Rev. 11/2005) Page 4-5

UNRESTRICTED

Purpose

This program performs a variable appraisal on a data file previously created by the user based oninformation gathered from an unrestricted random sample. Variable sampling is used to estimatequantitative characteristics. For each sampling unit the user obtains one or more numeric piecesof information about an event or item. The user has the option of obtaining and appraising fromone numeric piece of information per sample item (e.g., Examined amount) to as many as threepieces of information per sample item (i.e., Examined, Audited and Difference amounts). If theuser decides to appraise all three pieces of information, only two of the pieces of data may beentered and the third will be calculated by the program. The variable appraisal program assumesthat some variation exists between values. If no variation exists, then there is no need to run thisappraisal program.

Input Screen

VARIABLE - UNRESTRICTED RAT-STATS User’s Guide

Page 4-6 (Rev. 11/2005)



Universe size

The universe size is the total number of items from which the sampled items were selected. Thenumber should be entered without commas but upon exiting this box, the commas will beinserted. This number will be used in estimating universe parameters.

Specify input file

After entering the above information, click on Specify Input File. The following screen willappear. The input file format can be a text file, a table within an Access database, or an Excelspreadsheet. After selecting the desired format, click on Open Data File. Using the standardWindows “Open” file screen, locate the input file and double-click on it.

When the data file has been opened, the full input screen will appear (shown next).


(Rev. 11/2005) Page 4-7


Prior to executing this program, the user must create a data file that contains certain identifyingdata and one or two pieces of information for each sample unit selected. Each data line consistsof a line number for that sampling unit followed by the first piece of information (a numericvalue) the user wants to appraise (i.e., examined, audited, or difference value). If two or morepieces of information will be appraised and the examined amount is one of the values, then theexamined amount must be the first piece of data entered for each sampling unit. If only theaudited and difference amounts are being appraised, then the audited amount must be the firstpiece of data entered. The second piece of information may be the numeric difference betweenthe examined value and the amount accepted by the user or the audited amount if the examinedamount was the first piece of data entered.


Page 4-8 (Rev. 11/2005)


7483 289.99 43.00Explanation:

7483 - This is a number assigned by the user. The user should use the sample itemnumber as the number in this position. For ease of reference in this example, thenumber will be referred to as the line number.

289.99 - This is a number being reviewed by the user. The number, for example, could bea dollar amount claimed or the number of items on an inventory card. If thenumber is negative, then a minus sign must precede the number. The user mustinsert at least one space between the line number and the first numeric valueentered on the line.

43.00 - If two pieces of information are being gathered for each sampling unit, then thisnumber is the audited or difference amount determined by the user. For example,if the user had determined that of $289.99 claimed by a vendor, only $246.99 wasactually owed, then the difference amount entered would be $43.00.

The user must enter the data for each sampling unit on a separate line. For ease of editing,enough spaces or tabs should be placed between values so that the sample values align vertically. The data values may contain commas and dollar signs ($). The program assumes one or morespaces or tabs as the only delimiters between pieces of data.

Input From a Text File

The sample data may be stored in a text file format. There are several ways the user may createthis file. The easiest approach would be with a word processing package (e.g., Word) or a printfile created with a spreadsheet package (e.g., Excel).


(Rev. 11/2005) Page 4-9

NOTE: Example is for illustrative purposes only. The sample sizes may not conform to theorganization’s minimum sample size standards.

The first 20 rows and last three rows of data set DATASRS.TXT are shown below.

1 300 2672 900 7743 300 2554 200 174 Data set DATASRS.TXT5 900 8106 700 5607 1000 8208 100 809 900 76510 700 63011 700 63012 400 33213 300 25514 100 8415 200 16816 100 8817 600 52818 400 34019 900 74720 1000 800 . . .48 300 23749 500 43550 100 86

When the data file has been opened, the program will return to the input screen.


The sample data may be stored in a table within an Access database. Select the name of thedatabase containing the input table in the preceding Open Data File step. This database musthave the standard Access extension (.MDB). The name of the database for this illustration isC:\TEMP\VARIABLE.MDB. The name of the Access table within database VARIABLE.MDBfor this illustration is DATASRS. The following table shows the first 20 rows of Access tableDATASRS. The field name for the first column (“Line-Number” in the illustration) is arbitraryand is not used by the program at any point.


Page 4-10 (Rev. 11/2005)


After opening the database, the user will be asked to select the name of the table within theselected database using the following form. Click on the down arrow under Select a table. After selecting a table from the drop-down list (DATASRS for this illustration) and clicking onClick here to see field names, the form shown next will appear.


(Rev. 11/2005) Page 4-11

To select the field names, click on the field name for the field containing the first piece ofinformation in the input file (“Examined” in this illustration) and then click on the top right-handbox. The field name will then appear in this box. Repeat this for the field name of the secondfield in this table (“Audited”) and click on the middle right-hand box to specify this field name.


Page 4-12 (Rev. 11/2005)


When the field names have been specified, click on OK. The program will return to the inputscreen. NOTE: When the user returns to the input screen, the data file format (Examined andAudited Values for this illustration) will be selected based on responses within the precedingform to select Access Table and Field Names. When using an Access input file, the user cannotchange this data file format option after returning to the input screen.


The sample data can be stored in an Excel spreadsheet. Select the name of the spreadsheetcontaining the input table in the preceding Open Data File step. This file must have thestandard Excel extension (.XLS). For this illustration, Excel file DATASRS.XLS will be used. The first 20 rows of this file are shown next.

After opening the data file, the user is returned to the input screen. Click on CONTINUE toresume processing. When using the Excel option, the following screen will appear. Thisparticular file contains labels (variable names) in the first row and line numbers (1, 2, 3, ...) in


(Rev. 11/2005) Page 4-13

column A. The corresponding options were selected in the Excel Input screen. The linenumbers are optional. Had column A contained the examined values, the second option in theColumn A frame in the Excel Input screen should have been selected.

When the Excel Input information has been completed, click on OK. The program will resumeprocessing.

Output Options



Page 4-14 (Rev. 11/2005)

If the user selects a text file for output, the program will prompt for a file name. The standardWindows “Save” file screen will appear. The user should type in the file name in the designatedbox and click on Save. The output file will be saved with a “.TXT” extension.

If the user selects the printer for output, the standard Windows “Print” dialog box will appear. Select the printer to use for the program output.

After selecting the output format, click on CONTINUE. The next screen to appear is the datafile summary, shown next. At this point the user should reconcile the values to determine thatthe data file is complete and accurate. Click on OK to continue or EXIT to exit the program.

Program Output

For the examined, adjusted, and difference sections of the output, the following pieces ofinformation will be displayed. The precision information is given at the two-sided 80%, 90%,and 95% confidence levels.

MEAN The average value for the sample items appraised. It is obtained bysumming the items in the sample and dividing the result by the numberof items in the sample.

UNIVERSE The quantity of the items from which the sample was drawn. Theresults of the sample will be projected to the universe using this value.

STANDARDDEVIATION

A measurement of the variation of the sample items about the averagevalue (mean).

STANDARDERROR (MEAN)

A measurement of the variation of the estimated universe mean withrespect to all possible estimated means for this universe and samplesize.


(Rev. 11/2005) Page 4-15


A measurement of the variation of the estimated universe total withrespect to all possible estimated totals for this universe and sample size.

SKEWNESS A measure of the symmetry of the frequency distribution of the sampleitems. Accounting universes are usually right-skewed (majority ofitems have a low value while a few items have a high value).

KURTOSIS A measure of the peakedness or flatness of the frequency distribution ofthe sample values.

POINTESTIMATE

A single estimate for the universe total based on the sample meanmultiplied by the universe size.

CONFIDENCELEVEL

The confidence (80%, 90%, 95%) associated with the ability of the corresponding interval to contain the true mean (or universe total).

LOWER LIMIT The lower bound of the confidence interval derived by subtracting theprecision amount from the point estimate.

UPPER LIMIT The upper bound of the confidence interval derived by adding theprecision amount to the point estimate.

PRECISIONAMOUNT

A measurement of the closeness of the sample estimate of the universetotal and the corresponding unknown universe value. The precisionamount is calculated by multiplying the standard error by the universesize and multiplying the result by the appropriate factor (“t” value)corresponding to the desired confidence level.

PRECISIONPERCENT

The result of dividing the precision amount by the point estimate andstating the result as a percentage.

t-VALUE USED The t- percentile value used to construct the confidence interval.


Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTDISK.TXT, shown next. The printer output is identical.


Page 4-16 (Rev. 11/2005)

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 VARIABLE UNRESTRICTED APPRAISAL Time: 14:14 AUDIT/REVIEW: Variable SRS DATA FILE USED: C:\temp\DATASRS.TXT

SAMPLE EXAMINED NONZERO TOTAL OF TOTAL OF SIZE VALUE DIFFS DIFF VALUES AUD VALUES 50 24,800.00 50 3,530.00 21,270.00

----------------------- E X A M I N E D ------------------------ MEAN / UNIVERSE 496.00 10,000 STANDARD DEVIATION 296.90 SKEWNESS .32 KURTOSIS 1.81 STANDARD ERROR (MEAN) 41.88 STANDARD ERROR (TOTAL) 418,823 POINT ESTIMATE 4,960,000




----------------------- A U D I T E D -------------------------- MEAN / UNIVERSE 425.40 10,000 STANDARD DEVIATION 256.20 SKEWNESS .30 KURTOSIS 1.78 STANDARD ERROR (MEAN) 36.14 STANDARD ERROR (TOTAL) 361,412 POINT ESTIMATE 4,254,000



(Rev. 11/2005) Page 4-17




--------------------- D I F F E R E N C E ---------------------- MEAN / UNIVERSE 70.60 10,000 STANDARD DEVIATION 48.25 SKEWNESS .64 KURTOSIS 2.98 STANDARD ERROR (MEAN) 6.81 STANDARD ERROR (TOTAL) 68,068 POINT ESTIMATE 706,000





Page 4-18 (Rev. 11/2005)


Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedincludes a text file or printer. The following screen is the summary of the examined values forthis illustration. To see the summary for the audited values, click on Additional Summary Infoat the bottom of this form. To see the summary for the difference values, click again onAdditional Summary Info.

RAT-STATS User’s Guide VARIABLE - STRATIFIED

(Rev. 11/2005) Page 4-19

STRATIFIED

Purpose

This program performs a stratified variable appraisal on a data file previously created by the userbased on information gathered from a stratified random sample. Variable sampling is used toestimate quantitative characteristics. For each sampling unit the user obtains one or morenumeric pieces of information about an event or item. The user has the option of obtaining andappraising from one numeric piece of information per sample item (e.g., Examined amount) to asmany as three pieces of information per sample item (i.e., Examined, Audited and Differenceamounts). If the user decides to appraise all three pieces of information, only two of the piecesof data may be entered and the third will be calculated by the program. The variable appraisalprogram assumes that some variation exists between values.

The user normally selects stratification to improve sample efficiency. The area of interest, forexample, may be placed into segments (strata) based on value of items (e.g., high and low dollarvalue of transactions) or the sensitivity of items reviewed (e.g., entertainment and payroll costs). The program allows for a maximum of 50 strata to be appraised. The user must know theuniverse size of each stratum in order to use this methodology.

Input Screen

VARIABLE - STRATIFIED RAT-STATS User’s Guide

Page 4-20 (Rev. 11/2005)



Number of strata

After entering the name of the audit/review, the user must enter the number of strata. Themaximum number of strata is 50.

Specify input file

After entering the name and the number of strata, click on the Specify Input File button and thefollowing screen will appear. The input file(s) format can consist of two text files, two tableswithin an Access database, or two spreadsheets within the same Excel file.

After selecting the desired format, click on Open Data File. Using the standard Windows“Open” file screen, locate the input file and double-click on it.

When all files have been opened, the full input screen (shown next) will appear:


(Rev. 11/2005) Page 4-21


Prior to executing this program, the user must create a sample data file that contains certainidentifying data and one or two pieces of information for each sample unit selected within eachstratum. Each data line consists of a line number for that sampling unit followed by the firstpiece of information (a numeric value) the user wants to appraise (i.e., Examined, Audited, orDifference value). If two or more pieces of information will be appraised and the examinedamount is one of the values, then the examined amount must be the first piece of data entered foreach sampling unit. If only the audited and difference amounts are being appraised, then theaudited amount must be the first piece of data entered. The second piece of information may be


Page 4-22 (Rev. 11/2005)

the numeric difference between the examined value and the amount accepted by the user or theaudited amount if the examined amount was the first piece of data entered.

Regardless of the software used to create the sample data file, the format should be as follows:

7483 289.99 43.00Explanation:






The sample data and universe/sample size information may be stored in a text file format. Thereare several ways the user may create this file. The easiest approach would be with a wordprocessing package (e.g., Word) or a print file created with a spreadsheet package (e.g., Excel).

Sample Data File

A portion of data set DATASTRAT.TXT is shown below. The sample data file contains 25observations in each stratum.


(Rev. 11/2005) Page 4-23


1 80 2 43 3 133 4 125 5 116 . Sample Data file DATASTRAT.TXT . .21 12722 10523 10224 6925 7626 35427 32828 31329 25030 261 . . .46 29547 27748 35549 31450 277

After clicking on the Open Data File button and opening the sample data file (e.g.,DATASTRAT.TXT), the Open/Read Universe File button will appear (shown next).


Page 4-24 (Rev. 11/2005)

Universe/Sample Size File

To open the file containing the universe/sample sizes, click on the Open/Read Universe File button and use the standard Windows “Open” file screen to locate the universe file(e.g., UnivStrat.TXT--shown below). The format of this file is:

line counter, universe size, sample size

There is one line for each stratum in the sample. Values within a line can be separated by one ormore spaces or by using the tab key. Commas are allowed in the universe and sample sizevalues. After the universe file has been selected, the program will return to the input screen.

After opening the data files, the user is returned to the input screen. Click on CONTINUE toresume processing. The user will need to specify the column(s) containing the input sample dataand the total number of columns in the sample data file using the screen below:


(Rev. 11/2005) Page 4-25

After entering the column information, click on OK. The program will resume processing.


The sample data and universe/sample size information must be stored in two tables within thesame Access database. Select the name of the database containing the input tables in thepreceding Open Data File step. This database must have the standard Access extension(.MDB). The name of the database for this illustration is C:\TEMP\VARIABLE.MDB. Theuser may double-click on the VARIABLE database name or single-click on it and the Openbutton on the standard Windows “Open” file screen.

Universe/Sample Sizes Table

The name of the Access table containing the universe/sample size information for thisillustration is UnivStrat. The following table shows the table contents:


Page 4-26 (Rev. 11/2005)

Sample Data Table

The name of the Access table containing the sample data for this illustration is DATASTRAT. The following table shows the first five rows. The field name for the first column (“Line-Number” in the illustration) is arbitrary and is not used by the program at any point.

This sample data file contains 25 observation in each stratum. The last three rows of the firststratum and the first two rows of the second stratum are shown below:

After opening the database, the user will be asked to select the name of the table containing theuniverse/sample information and the table containing the sample data using the following form. For the universe/sample sizes table, click on the down arrow under Select the table. Afterselecting a table from the drop-down list (UnivStrat for this illustration), click on Click here tosee field names. This list will contain the field names for this table. To select the field names,click on the field name for the field containing the universe sizes (“Universe Size” in thisillustration) and click on the box labeled “Universe Sizes.” The field name will appear in thisbox. Repeat this procedure for the field containing the sample sizes (“Sample Size” in thisillustration).


(Rev. 11/2005) Page 4-27

To select the field names for the table containing the sample data, first select the table(DATASTRAT in this illustration), click on Click here to see field names, and click on the fieldname for the field containing the first piece of information in the input file (“Difference” in thisillustration) and click on the box labeled “Difference Values.” The field name will appear in thisbox. Repeat this procedure for any remaining field names in this table (there are none for thisillustration). When all the field names have been specified, click on OK. The program willreturn to the input screen. NOTE: When the user returns to the input screen, the data file format(Difference Values for this illustration) will be selected, based on responses within the preceding


Page 4-28 (Rev. 11/2005)

Access Table and Field Names window. When using an Access input file, the user cannotchange this data file format option after returning to the input screen.


With this option, the sample data and universe/sample sizes information must be stored in twoExcel spreadsheets within the same Excel file. Select the name of the file containing bothspreadsheets in the preceding Open Data File step. This file must have the standard Excelextension (.XLS). For this illustration, Excel file DATASTRAT.XLS will be used.

Universe/Sample Sizes Spreadsheet

The contents of the spreadsheet containing the universe/sample sizes (named Universe in thisillustration) are shown below:

Sample Data Spreadsheet

The name of the second spreadsheet containing the sample data is DATASTR in this illustration. The first five observations in this spreadsheet are shown below. The sample data file contains 25observations in each stratum for a total of 51 rows (including the first row with labels). Thisparticular file contains line numbers (1, 2, 3, . . .) in column A. The line numbers are optional.

After opening the data file, the user is returned to the input screen. Click on CONTINUE toresume processing. When using the Excel option, the following screen will appear. For thisillustration, the various boxes should be filled in as shown.


(Rev. 11/2005) Page 4-29


After entering the cell locations, click on OK. The program will resume processing.

Complete or Summary Output

The user may want to reduce printed output by having only the summary of the appraisal created. The default is for the complete appraisal output.


Page 4-30 (Rev. 11/2005)

Output Options




After selecting the output format, click on CONTINUE. The next screen to appear is the datafile summary, shown next. At this point the user should reconcile the values to determine thatthe sample data file is complete and accurate. Click on OK to continue or EXIT to exit theprogram.


(Rev. 11/2005) Page 4-31

Program Output

For the examined, audited, and difference sections of the output, the following pieces ofinformation will be displayed. The precision information is given at the two-sided 80%, 90%,and 95% confidence levels.

MEAN The average value for the sample items appraised. It is obtained bysumming the items in the sample and dividing the result by the numberof items in the sample.

UNIVERSE The quantity of the items from which the sample was drawn. Theresults of the sample will be projected to the universe using this value.

STANDARDDEVIATION

A measurement of the variation of the sample items about the averagevalue (mean).

STRATUMSTANDARDERROR (MEAN)

A measurement of the variation of the sample mean with respect to allpossible means for this stratum universe and this sample size.

STRATUMSTANDARDERROR(TOTAL)

A measurement of the variation of the estimated stratum universe totalwith respect to all possible estimated totals for this stratum universe andsample size.


A measurement of the variation of the point estimate of the total withrespect to all possible totals for this universe and these sample sizes.

SKEWNESS A measure of the symmetry of the frequency distribution of the sampleitems. Accounting universes are usually right-skewed (majority ofitems have a low value while a few items have a high value).

KURTOSIS A measure of the peakedness or flatness of the frequency distribution ofthe sample values.

POINTESTIMATE

A single estimate for the universe total based on the sample meanmultiplied by the universe size.

CONFIDENCELEVEL

The confidence (80%, 90%, 95%) associated with the ability of thecorresponding interval to contain the true mean (or universe total).


Page 4-32 (Rev. 11/2005)



STRATUMPRECISIONAMOUNT

A measurement of the closeness of the sample estimate of the universetotal and the corresponding unknown universe value. The precisionamount is calculated by multiplying the stratum standard error by thestratum universe size and multiplying the result by the appropriatefactor (“t” value) corresponding to the desired confidence level. For theexamined (reviewed) appraisal, the stratum total may be known andshould be reviewed by the user to see if, in fact, the actual value doesfall within the confidence interval.

OVERALLPRECISIONAMOUNT

A measurement of the closeness of the sample estimate of the universetotal and the corresponding unknown universe value. The precisionamount is calculated by multiplying the overall standard error by theappropriate factor (“Z” value) corresponding to the desired confidencelevel. For the examined (reviewed) appraisal, the universe total may beknown and should be reviewed by the user to see if, in fact, the actualvalue does fall within the confidence interval.

PRECISIONPERCENT


t-VALUE USED The t-percentile value used to construct the confidence interval.



Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTSTRAT.TXT, shown next. The printer output is identical.


(Rev. 11/2005) Page 4-33

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 STRATIFIED VARIABLE APPRAISAL Time: 15:18 AUDIT/REVIEW: Variable - Stratified

DATA FILE USED: C:\temp\DATASTRAT.TXT


TOTALS 50 10,266.00 50

--------------------- D I F F E R E N C E ----------------------Stratum 1 MEAN / UNIVERSE 99.24 5,200 STANDARD DEVIATION 26.33 SKEWNESS -.07 KURTOSIS 2.24 STANDARD ERROR (MEAN) 5.25 STANDARD ERROR (TOTAL) 27,319 POINT ESTIMATE 516,048




Stratum 2 MEAN / UNIVERSE 311.40 3,500 STANDARD DEVIATION 39.64 SKEWNESS -.06 KURTOSIS 1.85 STANDARD ERROR (MEAN) 7.90 STANDARD ERROR (TOTAL) 27,651 POINT ESTIMATE 1,089,900


Page 4-34 (Rev. 11/2005)





OVERALL POINT ESTIMATE / UNIVERSE 1,605,948 8,700 STANDARD ERROR 38,870





(Rev. 11/2005) Page 4-35

Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedincludes a text file or printer. The screen immediately following is the summary of thedifference values for the first stratum in this illustration. If the user created a sample data filecontaining two values per sample item (e.g., examined and audited values), the summary foreach amount (examined, audited, difference) can be obtained by clicking on AdditionalSummary Info at the bottom of this screen. For this illustration, the Additional Summary Infobutton is not visible since only the difference values were contained in the sample data file. Toobtain the results for the second stratum, click on Next Stratum. The user can click on NextStratum and Previous Stratum to review the results for the individual strata. To obtain theoverall results, click on OVERALL.


Page 4-36 (Rev. 11/2005)


RAT-STATS User’s Guide VARIABLE - TWO-STAGE UNRESTRICTED

(Rev. 11/2005) Page 4-37


Purpose

This program performs a two-stage variable appraisal on a data file previously created by theuser based on information gathered from a two-stage random sample. Variable sampling is usedto estimate quantitative characteristics. For each sampling unit the user obtains one or morenumeric pieces of information about an event or item. The user has the option of obtaining andappraising from one numeric piece of information per sample item (e.g., Examined amount) to asmany as three pieces of information per sample item (i.e., Examined, Audited, and Differenceamounts). If the user decides to appraise all three pieces of information, only two of the piecesof data may be entered and the third will be calculated by the program. The variable appraisalprogram assumes that some variation exists between values. If no variation exists, then there isno need to run the appraisal program.

Input Screen

NOTE: The Specify Input Information button only becomes visible when a value has beenspecified in the Number of Primary Units in Universe box.

VARIABLE - TWO-STAGE UNRESTRICTED RAT-STATS User’s Guide

Page 4-38 (Rev. 11/2005)



Number of primary units in sample

The user enters the number of primary units in the sample. The program will only accept anumeric entry.

Number of primary units in universe

The user enters the number of primary units in the universe. The program will only accept anumeric entry.

After entering the above information, click on Specify Input Information. The followingscreen will appear. The input file(s) format can consist of two text files, two tables within anAccess database, or two spreadsheets within the same Excel file. If the text file option isselected, a series of buttons will become visible. To specify the data file click on Open DataFile and to open the file containing the universe/sample sizes click on the Open/Read UniverseFile button. If either the Access File or Excel File option is selected, click on Open Data File tospecify the Access database or Excel file.


(Rev. 11/2005) Page 4-39

After selecting the desired format, click on Open Data File. Using the standard Windows“Open” file screen, locate the input file and double click on it.

When all files have been opened, the full input screen will appear.


Prior to executing this program, the user must create a sample data file that contains certainidentifying data and one or two pieces of information for each sample unit selected. Each data


Page 4-40 (Rev. 11/2005)

line consists of a line number for that sampling unit followed by the first piece of information (anumeric value) the user wants to appraise (i.e., Examined, Audited, or Difference value). If twoor more pieces of information will be appraised and the examined amount is one of the values,then the examined amount must be the first piece of data entered for each sampling unit. If onlythe audited and difference amounts are being appraised, then the audited amount must be the firstpiece of data entered. The second piece of information may be the numeric difference betweenthe examined value and the amount accepted by the user or the audited amount if the examinedamount was the first piece of data entered.


7483 289.99 43.00Explanation:





Input From Text Files

The sample data and universe/sample size information may be stored in a text file format. Thereare several ways the user may create this file. The easiest approach would be with a wordprocessing package (e.g., Word) or a print file created with a spreadsheet package (e.g., Excel).


(Rev. 11/2005) Page 4-41


Sample Data File

The sample data file used in this illustration (C:\TEMP\DATA2STG.TXT) is shown below andconsists of data from 10 sampled primary units selected from a total of 90 primary units. Thereare a total of 104 sample items.

Page 1 Page 2 Page 3 Page 4 Page 5

1 52 73 94 05 116 27 88 49 310 511 412 313 714 215 1116 017 118 919 420 321 2

22 123 524 525 626 427 1128 1229 030 131 832 433 634 435 036 137 038 939 840 441 642 10

431144445346147048249850651552353125411553564572580590601614623632

64 465 366 767 668 769 870 471 372 273 374 675 476 377 278 279 880 481 082 483 584 6

85 386 687 488 789 390 991 192 493 594 695 796 597 1098 1199 2100 1101 4102 0103 5104 4

After clicking on the Open Data File button and opening the sample data file (e.g.,DATA2STG.TXT), the Open/Read Universe File button will appear on the following screen.


Page 4-42 (Rev. 11/2005)

Universe/Sample Size File

To open the file containing the universe/sample sizes, click on the Open/Read Universe Filebutton and use the standard Windows “Open” file screen to locate the universe file(e.g., UNIV2STG.TXT, shown below). The format of this file is:


There is one line for each primary unit in the sample. Values within a line can be separated byone or more spaces or by using the tab key. Commas are allowed in the universe and samplesize values.


(Rev. 11/2005) Page 4-43

After selecting the universe file and clicking on OK, the program will return to the input screen. Click on CONTINUE to resume processing. The user will need to specify the column(s)containing the input data and the total number of columns in the sample data file using the screenbelow:



The sample data and universe/sample size information must be stored in two tables within thesame Access database. Select the name of the database containing the input tables in thepreceding Open Data File step. This database must have the standard Access extension(.MDB). The name of the database for this illustration is C:\TEMP\VARIABLE.MDB. Theuser may double-click on the VARIABLE database name or single-click on it and the Openbutton.

Universe/Sample Size Table

The name of the Access table containing the universe/sample size information for thisillustration is Univ2Stg. The following table shows the table contents.


Page 4-44 (Rev. 11/2005)

NOTE: Example is for illustrativepurposes only. The sample sizes maynot conform to the organization’sminimum sample size standards.

Sample Data Table

The first 14 rows of the Access table containing the sample data DATA2STG are shown next. The field name for the first column (“Line” in the illustration) is arbitrary and is not specified orused by the program at any point. For a view of the complete file, refer to the previous InputFrom a Text File section.

After opening the database, the user will be asked to select the name of the table containing theuniverse/sample information and the table containing the sample data using the following form. For the universe/sample sizes table, click on the down arrow under Select the table. Afterselecting a table from the drop-down list (Univ2Stg for this illustration), click on Click here to


(Rev. 11/2005) Page 4-45

see field names. This list will contain the field names for this table. To select the field names,click on the field name for the field containing the universe sizes (“Universe Size” in thisillustration) and click on the box labeled “Universe Sizes.” The field name will appear in thisbox. Repeat this procedure for the field containing the sample sizes (“Sample Size” in thisillustration).

To select the field name(s) for the sample data, first select the table (DATA2STG in thisillustration), click on Click here to see field names, and click on the field name for the fieldcontaining the first piece of information in the input file (“Difference” in this illustration) andthen click on the box labeled “Difference.” The field name will then appear in this box. Repeat


Page 4-46 (Rev. 11/2005)

this procedure for any remaining field names in this table (there are none for this illustration). When all the field names have been specified, click on OK. The program will return to the inputscreen. NOTE: When the user returns to the input screen, the data file format (DifferenceValues for this illustration) will be selected, based on responses within the preceding form toselect Access Table and Field Names. When using an Access input file, the user cannot changethis data file format option after returning to the input screen.

Input From Excel Spreadsheets

With this option, the sample data file and universe/sample sizes information must be stored intwo Excel spreadsheets within the same Excel file. Select the name of the file containing bothspreadsheets in the preceding Open Data File step. This file must have the standard Excelextension (.XLS).

Universe/Sample Size Spreadsheet

For this illustration, Excel file DATA2STG.XLS will be used. The contents of the spreadsheetcontaining the universe/sample sizes (named Universe in this illustration) are shown below:


The name of the second spreadsheet containing the sample data is DATA2STG in thisillustration. The first 15 observations in this file are shown next. For a view of the completefile, refer to the previous Input From a Text File section. This particular file contains linenumbers (1, 2, 3, . . .) in column A. The line numbers are optional.


(Rev. 11/2005) Page 4-47

NOTE: Example is for illustrativepurposes only. The sample sizes maynot conform to the organization’sminimum sample size standards.

After opening the data file, the user is returned to the input screen. Click on CONTINUE toresume processing. When using the Excel option, the following screen will be displayed. Forthis illustration, the various boxes should be filled in as shown. After entering the cell locations,click on OK. The program will resume processing.


Page 4-48 (Rev. 11/2005)

Specifying the Number of Secondary Units

The user enters the total number of secondary units for all of the primary units in the universe. The program will only accept a numeric entry. If the number of secondary units is not known,click on Unknown. For this case, the output will not display the quantity of secondary units inthe universe.


(Rev. 11/2005) Page 4-49

Output Options




After selecting the output format, click on CONTINUE. The next thing to appear is the data filesummary, shown next. At this point the user should reconcile the values to determine that thedata file is complete and accurate. Click on OK to continue or EXIT to exit the program.


Page 4-50 (Rev. 11/2005)

Program Output

For the examined, adjusted, and difference sections of the output, the following pieces ofinformation will be displayed. The precision information is given at the two-sided 80%, 90%,and 95% confidence levels.

UNIT NBR The indicator of the sampled primary unit that is being appraised.

SAMPLE SIZE The number of items sampled in the particular primary unit.

NONZEROITEMS

The number of nonzero items sampled in the particular primary unit.

SAMPLE MEAN The average value for the sample items appraised. It is obtained bysumming the items in the sample and dividing the result by the numberof items in the sample.

VARIANCE A measurement of the variation of the sampled items about the samplemean.

UNIVERSE SIZE The quantity of the secondary items within each sampled primary unitfrom which the sample was drawn. The results of the sample will beprojected to the primary unit universe using this value.

POINTESTIMATE

The single estimate for the universe total of the primary unit based onthe sample mean and universe size.

STANDARDERROR

A measurement of the variation of the overall point estimate of theuniverse total with respect to all possible point estimates for thisuniverse and these sample sizes.

CONFIDENCELEVELS

The confidence (80%, 90%, or 95%) associated with the ability ofcorresponding interval to contain the true mean (or universe total).




(Rev. 11/2005) Page 4-51

PRECISIONAMOUNT

A measurement of the closeness of the sample estimate of the universetotal and the corresponding unknown universe value. The precisionamount is calculated by multiplying the standard error by theappropriate factor (“Z” value) corresponding to the desired confidencelevel. For the examined (reviewed) appraisal, the universe total may beknown and should be reviewed by the user to see if, in fact, the actualvalue does fall within the confidence interval.

PRECISIONPERCENT


Z-VALUE USED The value referred to as the standard normal deviate. It is ameasurement from the point estimate to a confidence limit measured instandard errors.


Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUT2STG.TXT, shown next. The printer output is identical.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 TWO-STAGE UNRESTRICTED VARIABLE APPRAISAL Time: 13:47 AUDIT/REVIEW: Variable 2-Stage

DATA FILE USED: C:\temp\DATA2STG.TXT


1 10/9 5.40 11.38 50 270 2 13/12 4.00 10.67 65 260 3 9/8 5.67 16.75 45 255 4 10/8 4.80 13.29 48 230 5 10/9 4.30 11.12 52 224 6 12/10 3.83 14.88 58 222 7 8/8 5.00 5.14 42 210 8 13/12 3.85 4.31 66 254 9 8/8 4.88 6.13 40 195 10 11/10 5.00 11.80 56 280

104/94 4.80 522 2,400


Page 4-52 (Rev. 11/2005)



STANDARD ERROR 867




Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedis a text file or printer. The screen below is the summary of the difference values for the firstprimary unit in this illustration. If the user created a sample data file containing two values persample item (e.g., examined and audited values), the summary for each amount (examined,audited, difference) can be obtained by clicking on Additional Summary Info at the bottom ofthis form. For this illustration, the Additional Summary Info button is not visible since onlythe difference values were used in the data file. To obtain the results for the second primary unit,click on Next Primary Unit. The user can click on Next Primary Unit and Previous PrimaryUnit to review the results for the individual primary units. To obtain the overall results, click onOVERALL (shown next, immediately following the summary screen for the first primary unit).


(Rev. 11/2005) Page 4-53


Page 4-54 (Rev. 11/2005)


RAT-STATS User’s Guide VARIABLE - THREE-STAGE UNRESTRICTED

(Rev. 11/2005) Page 4-55


Purpose

This program performs a three-stage variable appraisal on a data file previously created by theuser based on information gathered from a three-stage random sample. Variable sampling isused to estimate quantitative characteristics. For each sampling unit the user obtains one or morenumeric pieces of information about an event or item. The user has the option of obtaining andappraising from one numeric piece of information per sample item (e.g., Examined amount) to asmany as three pieces of information per sample item (i.e., Examined, Audited, and Differenceamounts). If the user decides to appraise all three pieces of information, only two of the piecesof data may be entered and the third will be calculated by the program. The variable appraisalprogram assumes that some variation exists between values. If no variation exists, then there isno need to run the appraisal program.

Input Screen

NOTE: The Specify Input Information button only becomes visible when the number ofprimary units in the universe and the sample have been entered.

VARIABLE - THREE-STAGE UNRESTRICTED RAT-STATS User’s Guide

Page 4-56 (Rev. 11/2005)



Number of primary units in the universe

The user enters the number of primary units in the universe. This quantity may be entered withcommas and the program will only accept a numeric entry.

Number of primary units in the sample

The user enters the number of primary units in the sample. This quantity may be entered withcommas; the program will only accept a numeric entry.

After entering the above information, click on Specify Input Information. The followingscreen will appear. The format can consist of three text files, three tables within the sameAccess database, or three spreadsheets within the same Excel file. If the text file option isselected, a series of buttons will become visible (one at a time) and the user will need to click onOpen Data File to specify the data file, click on the Open Primary Unit File button to specifythe file containing information on the primary units, then click on the Open Secondary UnitFile button to specify the file containing information on the secondary units. If either the AccessFile or Excel File option is selected, click on Open File to specify the Access database or Excelfile.


(Rev. 11/2005) Page 4-57



Prior to executing this program, the user must create a sample data file that contains certainidentifying data and one or two pieces of information for each sample unit selected. Each dataline consists of a line number for that sampling unit followed by the first piece of information (anumeric value) the user wants to appraise (i.e., Examined, Audited, or Difference value). If twoor more pieces of information will be appraised and the examined amount is one of the values,


Page 4-58 (Rev. 11/2005)

then the examined amount must be the first piece of data entered for each sampling unit. If onlythe audited and difference amounts are being appraised, then the audited amount must be the firstpiece of data entered. The second piece of information may be the numeric difference betweenthe examined value and the amount accepted by the user or the audited amount if the examinedamount was the first piece of data entered.

Format of Primary Unit Information

Regardless of the software used to create the primary unit file (a text file, Access table, or Excelspreadsheet), each line in the this file must contain at least three variables (fields). One possibleformat is the following:

REGION 1 10 2

Explanation:

REGION 1 - This is a description of a primary unit that was sampled. The description maybe up to 25 characters in length and may include spaces, commas, and otherspecial characters.

10 - This is the universe size of secondary units within this primary unit.

2 - This is the number of secondary units sampled from this primary unit.

Primary Unit File Restrictions

1. The primary unit description, the universe size of secondary units, and the number ofsampled secondary units must be the final three entries in each line.

2. The user can enter one or more values preceding the primary unit description. Each valuemust be an integer, a space, or a tab. For example, the user may elect to begin each line witha counter (1, 2, 3, . . .). A counter will be used in the illustration to follow.

Format of Secondary Unit Information

Regardless of the software used to create the secondary unit file (a text file, Access table, orExcel spreadsheet), each line in the this file must contain at least three variables (fields). Onepossible format is the following:


(Rev. 11/2005) Page 4-59

MAINE 100 10

Explanation:

MAINE - This is a description of a secondary unit that was sampled. The descriptionmay be up to 25 characters in length and may include spaces, commas, andother special characters.

100 - This is the universe size of third-stage items within this secondary unit.

10 - This is the sample size of third-stage items within this secondary unit.

Secondary Unit File Restrictions

1. The secondary unit description, the universe size of third-stage items, and the number ofsampled third-stage items must be the final three entries in each line.

2. The user can enter one or more values preceding the secondary unit description. Eachvalue must be an integer, a space, or a tab. For example, the user may elect to begin eachline with a counter (1, 2, 3, . . .), a value identifying the primary unit number in thesecond column, and a value identifying the secondary unit number within each primaryunit in the third column. A counter and primary/secondary unit identifiers will be used inthe illustration to follow.

Format of Sample Data Information

Regardless of the software used to create the primary unit file (a text file, Access table, or Excelspreadsheet), each line in the this file must contain at least one variable (field) if each line in thesample data file contains a single piece of information or at least two variables (fields) if eachline in the sample data file contains two pieces of information. One possible format is thefollowing:

120.34 85.50

Explanation:

120.34 - This is the quantity being reviewed by the user. The number, for example,could be the value of a voucher or the quantity of items on an inventory card. If the quantity is negative, then a minus sign must precede the number. Theuser must insert at least one space between numeric values.


Page 4-60 (Rev. 11/2005)

85.50 - If two pieces of information are being gathered for each sampling unit, thenthis number is the audited or difference amount determined by the user. Forexample, if the user had determined that of $120.34 claimed by a vendor only$34.84 was actually owed, then the difference amount entered would be$85.50. The user must insert at least one space between the numeric values.


Sample Data File Restrictions

1. If each line contains a single piece of information (e.g., a difference value), this value mustbe the final entry. If each line contains two pieces of information (e.g., an examined amountand an audited amount), theses values must be the final two entries.

2. The user can enter one or more values preceding the sample data value(s). Each value mustbe an integer, a space, or a tab. For example, the user may elect to begin each line with a counter (1, 2, 3, . . .), a value identifying the primary unit number in the second column, avalue identifying the secondary unit number within each primary unit in the third column,and a value identifying the third-stage unit number within each sampled primary/secondaryunit in the fourth column. A counter and primary/secondary/third-stage unit identifiers willbe used in the illustration to follow.


The three data files containing the above information may be stored in a text file format. Thereare several ways the user may create this file. The easiest approach would be with a wordprocessing package (e.g., Word) or a print file created with a spreadsheet package (e.g., Excel).

Sample Data File

The sample data file in this illustration (C:\TEMP\DATA3STG.TXT) consists of the examinedamounts for three primary units (REGION 1, REGION 2, AND REGION 3) with 2, 1, and 2secondary units, respectively, selected from the three sampled primary units. Ten third-stageunits are sampled from each selected secondary unit.

The sample data file used here (C:\TEMP\DATA3STG.TXT) is shown next. Notice that this filecontains a counter (1, 2, 3, . . .) in the first column, a value identifying the primary unit numberin the second column, a value identifying the secondary unit number within each primary unit inthe third column, and a value identifying the third-stage unit number within each sampled


(Rev. 11/2005) Page 4-61

primary/secondary unit in the fourth column. These columns are not required but arerecommended for ease of navigating through this file.

1 1 1 1 1032 1 1 2 973 1 1 3 944 1 1 4 1005 1 1 5 936 1 1 6 1027 1 1 7 1048 1 1 8 1049 1 1 9 10310 1 1 10 10011 1 2 1 15212 1 2 2 15213 1 2 3 14714 1 2 4 16115 1 2 5 14416 1 2 6 15317 1 2 7 15418 1 2 8 15119 1 2 9 14820 1 2 10 15821 2 1 1 16722 2 1 2 17823 2 1 3 18224 2 1 4 17725 2 1 5 18326 2 1 6 18127 2 1 7 18028 2 1 8 18029 2 1 9 18130 2 1 10 19131 3 1 1 21432 3 1 2 20633 3 1 3 22834 3 1 4 20135 3 1 5 20636 3 1 6 21137 3 1 7 19438 3 1 8 20639 3 1 9 22440 3 1 10 21041 3 2 1 24042 3 2 2 24943 3 2 3 24644 3 2 4 254


Page 4-62 (Rev. 11/2005)


45 3 2 5 24746 3 2 6 25347 3 2 7 25348 3 2 8 24549 3 2 9 24850 3 2 10 255

Primary Unit File

The primary unit file used here (C:\TEMP\PRIMARY3STG.TXT) is shown below. Notice thatthis file contains a counter (1, 2, 3, . . .) in the first column. This column is not required.

1 Region 1 10 22 Region 2 8 13 Region 3 15 2

Secondary Unit File

The secondary unit file used here (C:\TEMP\SECONDARY3STG.TXT) is shown next. Noticethat this file contains a counter (1, 2, 3, . . .) in the first column, a value identifying the primaryunit number in the second column, and a value identifying the secondary unit number withineach primary unit in the third column. These columns are not required but are recommended forease of navigating through this file.

1 1 1 Maine 100 102 1 2 Vermont 125 103 2 1 New York 80 104 3 1 Texas 140 105 3 2 Arkansas 85 10

After opening the three text files, the program will return to the input screen. Click onCONTINUE to resume processing. The user will need to specify the column(s) containing theinput data and the total number of columns in the sample data file using the screen below.


(Rev. 11/2005) Page 4-63



The necessary information may be stored in three tables within the same Access database. Selectthe name of the database containing the input table in the preceding Open Data File step. Thisdatabase must have the standard Access extension (.MDB). The name of the database for thisillustration is C:\TEMP\VARIABLE.MDB. The user may double-click on the VARIABLEdatabase name or single-click on it and the Open button.

Primary Unit Table

The primary unit table used here (Primary3Stg) is shown below. Notice that this file contains acounter (1, 2, 3, . . .) in the first field. This field is not required.


Page 4-64 (Rev. 11/2005)

Secondary Unit Table

The secondary unit table used here (Secondary3Stg) is shown below. Notice that this filecontains a counter (1, 2, 3, . . .) in the first field, a value identifying the primary unit number inthe second field, and a value identifying the secondary unit number within each primary unit inthe third field. These fields are not required but are recommended for ease of navigating throughthis file.

Sample Unit Table

The first 10 rows of the data table used here (Data3Stg) is shown below. Notice that this filecontains a counter (1, 2, 3, . . .) in the first field, a value identifying the primary unit number inthe second field, a value identifying the secondary unit number within each primary unit in thethird field, and a value identifying the third-stage unit number within each sampledprimary/secondary unit in the fourth field. These fields are not required but are recommendedfor ease of navigating through this file.


(Rev. 11/2005) Page 4-65

After opening the database, the user will be asked to select the name of the table containing theprimary unit information, the table containing the secondary unit information, and the tablecontaining the sample data using the following form. For the primary unit table, click on thedown arrow under Select the table. After selecting a table from the drop-down list(Primary3Stg for this illustration), click on Click here to see field names. This list will containthe field names for this table. To select the field names, click on the field name for the fieldcontaining the primary unit identification (“Primary Unit Description” in this illustration) andclick on the box labeled “Primary Unit ID.” The field name will appear in this box. Repeat thisprocedure for the two fields containing the universe and sample sizes.

For the secondary unit table, click on the down arrow under Select the table. After selecting atable from the drop-down list (Secondary3Stg for this illustration), click on Click here to seefield names. This list will contain the field names for this table. To select the field names, clickon the field name for the field containing the secondary unit identification (“SSU Description” inthis illustration) and click on the box labeled “Secondary Unit ID.” The field name will appearin this box. Repeat this procedure for the two fields containing the universe and sample sizes foreach secondary unit.

To select the field name(s) for the sample data file, first select the table (Data3Stg in thisillustration), click on Click here to see field names, click on the field name for the fieldcontaining the first piece of information in the input file (“Examined” in this illustration), andthen click on the box labeled “Examined.” The field name will then appear in this box. Repeatthis procedure for any remaining field names in this table (there are none for this illustration). When all the field names have been specified, click on OK. The program will then return to theinput screen. NOTE: When the user returns to the input screen, the data file format (ExaminedValues for this illustration) will be selected, based on responses within the following SelectAccess Table and Field Names form. When using an Access input file, the user cannot changethis data file format option after returning to the input screen.


Page 4-66 (Rev. 11/2005)



With this option, the primary unit information, the secondary unit information, and the sampledata must be stored in three Excel spreadsheets within the same Excel file. Select the name ofthe file containing the spreadsheets in the preceding Open Data File step. This file must have


(Rev. 11/2005) Page 4-67

the standard Excel extension (.XLS). For this illustration, Excel file DATA3STG.XLS will beused.

Primary Unit Spreadsheet

The contents of the spreadsheet containing the primary unit information (Primary_Info in thisillustration) is shown below. Notice that this file contains a counter (1, 2, 3, . . .) in the firstcolumn. This column is not required.

Secondary Units Spreadsheet

The contents of the spreadsheet containing the secondary unit information (Secondary_Info inthis illustration) are shown next. Notice that this file contains a counter (1, 2, 3, . . .) in the firstcolumn, a value identifying the primary unit number in the second column, and a valueidentifying the secondary unit number within each primary unit in the third column. Thesecolumns are not required but are recommended for ease of navigating through this file.


The first 10 observations in the sample data table (Sample_Results) are shown next. Notice thatthis file contains a counter (1, 2, 3, . . .) in the first column, a value identifying the primary unitnumber in the second column, a value identifying the secondary unit number within eachprimary unit in the third column, and a value identifying the third-stage unit number within eachsampled primary/secondary unit in the fourth column. These columns are not required but arerecommended for ease of navigating through this file.


Page 4-68 (Rev. 11/2005)



(Rev. 11/2005) Page 4-69



Output Options

The output options are: (1) a text file and screen, (2) a printer and screen, (3) a text file, printerand screen, or (4) screen only. The program always concludes with a summary on the screen.



After selecting the output format, click on CONTINUE.

Program Output

If the complete option was selected by the user, the following information will be displayed forthe examined, adjusted, and difference sections of the output. The precision information is givenat the two-sided 80%, 90%, and 95% confidence levels.


Page 4-70 (Rev. 11/2005)

FIRST STAGE /SECOND STAGE

The description of the first- or second-stage unit as shown in the sampledata file.

SAMPLE SIZE The number of items sampled in the particular second stage.

NONZEROES The quantity of sample items in the particular second stage that hadvalues other than zero.

SAMPLE MEAN The average value for the sample items appraised. It is obtained bysumming the items in the sample and dividing the result by the numberof items in the sample.

VARIANCE A measurement of the variation of the sampled items about the samplemean.

UNIVERSE The quantity of the third-stage items within each sampled second-stageunit from which the sample was drawn.

POINTESTIMATE

The single estimate for the universe total of the second stage unit basedon the sample mean and universe.

If the user selects the summary option the results will be displayed only at the first-stage level. The universe of third-stage items and the variance at each second stage are not included in thesummary output.

The following information will be displayed in both the complete and summary output:

UNIVERSE For each stage, the number of items in the respective universes. For thesecond and third-stages, the universes represent the total for the prior-stage sample. For example, if 10 primary units are sampled from auniverse of 100 units, then the secondary universe is the total ofsecondary items in the 10 primary units reviewed and not the total ofsecondary units in the 100 primary units.

SAMPLED The number of units sampled at each stage.

OVERALLPOINTESTIMATE

The estimate of the universe total.


(Rev. 11/2005) Page 4-71


A measurement of the variation of the overall point estimate of theuniverse total with respect to all possible point estimates for thisuniverse and these sample sizes.

CONFIDENCELEVELS

The confidence associated with the ability of corresponding interval tocontain the true mean (or universe total).



PRECISIONAMOUNT

A measurement of the closeness of the sample estimate of the universetotal and the corresponding unknown universe value. The precisionamount is calculated by multiplying the standard error by theappropriate factor (“Z” value) corresponding to the desired confidencelevel. For the examined (reviewed) appraisal, the universe total may beknown and should be reviewed by the user to see if, in fact, the actualvalue does fall within the confidence interval.

PRECISIONPERCENT




Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUT3STG.TXT, shown immediately following. The printer output is identical.


Page 4-72 (Rev. 11/2005)

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 THREE-STAGE UNRESTRICTED VARIABLE APPRAISAL Time: 17:03 AUDIT/REVIEW: Variable 3-Stage

DATA FILE USED: C:\temp\Data3stg.TXT

----------------------- E X A M I N E D ------------------------FIRST STAGE SAMPLE NON- SECOND STAGE SIZE ZEROES SAMPLE MEAN VARIANCE UNIVERSE POINT ESTIMATE================ ====== ====== =========== ======== ========== ==============REGION 1 MAINE 10 10 100.00 16.44 100 10,000 VERMONT 10 10 152.00 25.33 125 19,000

COMBINED 20 14,500 10 145,000

REGION 2 NEW YORK 10 10 180.00 35.33 80 14,400

COMBINED 10 14,400 8 115,200

REGION 3 TEXAS 10 10 210.00 102.44 140 29,400 ARKANSAS 10 10 249.00 22.67 85 21,165

COMBINED 20 25,283 15 379,238

STAGES UNIVERSE SAMPLED FIRST 15 3 SECOND 33 5 THIRD 530 50

OVERALL POINT ESTIMATE 3,197,188OVERALL STANDARD ERROR 1,131,084

CONFIDENCE LIMITS 80% CONFIDENCE LEVEL LOWER LIMIT 1,747,646 UPPER LIMIT 4,646,729 PRECISION AMOUNT 1,449,542 PRECISION PERCENT 45.34% Z-VALUE USED 1.281551565545

90% CONFIDENCE LEVEL LOWER LIMIT 1,336,721 UPPER LIMIT 5,057,654 PRECISION AMOUNT 1,860,467 PRECISION PERCENT 58.19% Z-VALUE USED 1.644853626951


(Rev. 11/2005) Page 4-73


95% CONFIDENCE LEVEL LOWER LIMIT 980,304 UPPER LIMIT 5,414,071 PRECISION AMOUNT 2,216,883 PRECISION PERCENT 69.34% Z-VALUE USED 1.959963984540

Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedis a text file or printer. The screen below is the summary of the examined values for the firstprimary unit and the first secondary unit (REGION 1, MAINE) in this illustration. If the usercreated a sample data file containing two values per sample item (e.g., examined and auditedvalues), the summary for each amount (examined, audited, difference) can be obtained byclicking on Additional Summary Info at the bottom of this form. For this illustration, theAdditional Summary Info button is not visible since only the difference values were used in thesample data file.

By clicking on NEXT STAGE, the user can see the number of primary units (Stage-1) in theuniverse and in the sample, secondary units (Stage-2) in the universe and in the sample, andthird-stage units (Stage-3) in the universe and in the sample. Using the screen below, for thisillustration, there are 15 primary units in the universe and 3 in the sample. Based on thepreceding output to text file and printer, for Stage-2 the values would be 33 (universe) and 5(sample). For Stage-3, the values would be 530 (universe) and 50 (sample).


Page 4-74 (Rev. 11/2005)

To view summary information for the next sampled secondary unit within the selected primaryunit, click on Next in the Secondary Unit box. By clicking on Next and Previous, the user cannavigate through the secondary units for the selected primary unit. By clicking on Combined,the screen will contain the summary for the selected primary unit. Similarly, by clicking onNext and Previous in the Primary Unit box, the user can navigate through the sampled primaryunits. For each selected primary unit, the user can then switch to the Secondary Unit box andexamine the results for each secondary unit within this primary unit. Finally, by clicking onOverall in the Primary Unit box, the summary appraisal results for the entire analysis areshown. This output screen is shown next.


(Rev. 11/2005) Page 4-75


VARIABLE - RHC TWO STAGE RAT-STATS User’s Guide

Page 4-76 (Rev. 11/2005)

RHC TWO STAGE

Purpose

This program performs a two-stage variable appraisal using the Rao-Hartley-Cochran (RHC)methodology. This multistage procedure determines a confidence interval that uses approximate probability proportional to size (pps) sampling whereby the relative sizes of the sampling unitsare considered when selecting primary units to include in the sample. This is a variablesampling procedure allowing the user to obtain one or more quantitative pieces of informationabout an event or item. The user has the option of obtaining and appraising from one numericpiece of information per sample item (e.g., Examined amount) to as many as three pieces ofinformation per sample item (i.e., Examined, Audited, and Difference amounts).

Input Screen

NOTE: The Specify Input Information button only becomes visible when the number ofprimary units in the sample has been entered.

RAT-STATS User’s Guide VARIABLE - RHC TWO STAGE

(Rev. 11/2005) Page 4-77



To enter input information, click on Specify Input Information. The form at the bottom of thepage will appear. The sample data and primary unit (P.U.) file format can consist of two textfiles, two tables within the same Access database, or two spreadsheets within the same Excelfile. After selecting the desired format for the sample data file, click on Open data file. Note: Ifthe Access File (.MDB) option is selected, this button will change to Open Access database andif the Excel file (.XLS) option is selected, this button will change to Open Excel file. Using thestandard Windows “Open” file screen, locate the sample data file and double-click on it.

If the Text files option is being used, an Open Primary Unit File button will appear in the formbelow after the sample data file has been opened. The primary unit file is a slightly modifiedversion of the file created by the RHC Sample Selection program. Click on the Open PrimaryUnit File button and, using the standard Windows “Open” file screen, locate the primary unitfile and double click on it.



Page 4-78 (Rev. 11/2005)


Prior to executing this program, the user must create a sample data file that contains certainidentifying data and one or two pieces of information for each sample unit selected. Each dataline consists of a line number for that sampling unit followed by the first piece of information (anumeric value) the user wants to appraise (i.e., Examined, Audited, or Difference value). If twoor more pieces of information will be appraised and the examined amount is one of the values,then the examined amount must be the first piece of data entered for each sampling unit. If only


(Rev. 11/2005) Page 4-79

the audited and difference amounts are being appraised, then the audited amount must be the firstpiece of data entered. The second piece of information may be the numeric difference betweenthe examined value and the amount accepted by the user or the audited amount if the examinedamount was the first piece of data entered.


7483 289.99 43.00

Explanation:


289.99 - This is a number being reviewed by the user. The number, for example, could bea dollar amount claimed or the number of items on an inventory card. If thenumber is negative, then a minus sign must precede the number.

43.00 - If two pieces of information are gathered for each sampling unit, then this numberis the audited or difference amount determined by the user. For example, if theuser had determined that of $289.99 claimed by a vendor, only $246.99 wasactually owed, then the difference amount entered would be $43.00.


Format of Primary Unit Information

When the RHC Sample Selection program is used, a file is created containing information on theclustering of the primary units in the universe as well as the primary units selected. It may be atext file, a table within an Access database, or an Excel spreadsheet. This is an easy file to createsince the user only needs to insert the sample sizes (the third column, denoted as “15” in the linebelow) into the file created by the RHC Sample Selection program.

Regardless of the software used to create the primary unit file, the format should be as follows:

PRIMARY #1 200 15 1,100 12,500 8


Page 4-80 (Rev. 11/2005)

Explanation:

PRIMARY #1 - This is a description of the primary unit. A maximum of 30 characters,including spaces, will be accepted.

200 - This is the number of secondary units in the universe for this particularprimary unit. The user may include commas in the value.

15 - This is the number of secondary units in the sample for this particular primaryunit. The user may include commas in the value.

1,100 - This is the size factor assigned by the user to the primary unit (e.g., squarefootage, number of transactions, etc.). Commas and decimal points may beincluded in the values.

12,500 - This is the total size factor for the group in which the primary unit wasrandomly placed. Commas and decimal points may be included in the values.

8 - This is the quantity of primary units that were randomly placed into thisgroup. Commas may be included in the value.

The user must enter the data for each sampling unit on a separate line. For ease of editing,enough spaces or tabs should be placed between values so that the sample values align vertically. The program assumes one or more spaces or tabs as the only delimiters between pieces of data.


The sample data and primary unit information may be stored in a text file format. There areseveral ways the users may create this file. The easiest approach would be with a wordprocessing package (e.g., Word) or a print file created with a spreadsheet package (e.g., Excel).

Sample Data File

As an illustration, 10 universities (primary units) were selected using the RHC Sample Selectionprogram. The file below (C:\TEMP\DATARHC2.TXT) was obtained by sampling the 10selected universities.


(Rev. 11/2005) Page 4-81

1 9 μ P.U. 1 starts here2 23 94 65 06 57 78 29 710 6 μ P.U. 2 starts here11 012 613 014 315 416 117 1318 819 020 621 1122 823 824 025 0 μ P.U. 3 starts here26 627 028 729 230 031 932 933 534 035 536 837 1038 5 μ P.U. 4 starts here39 8

40 941 1142 243 844 245 146 747 348 449 050 651 1 μ P.U. 5 starts here52 553 454 055 656 557 258 759 2 μ P.U. 6 starts here60 661 862 963 764 465 166 1067 1168 069 170 671 172 2 μ P.U. 7 starts here73 174 675 176 577 1078 11


Page 4-82 (Rev. 11/2005)


79 980 1381 282 283 784 985 086 6 μ P.U. 8 starts here87 588 1089 690 1091 294 12 92 293 095 596 097 898 299 7100 7 μ P.U. 9 starts here101 10102 2103 6

104 0105 8106 4107 0108 10109 3110 2111 5112 10113 0114 0 μ P.U. 10 starts here115 0116 0117 0118 8119 9120 0121 2122 8123 4124 6125 2126 1127 8128 11

Primary Units File

For this illustration, there are 10 primary units (universities). The modified file(PRIMRHC2.TXT) created by the RHC Sample Selection program (the primary unit file) isshown below. The sample sizes (highlighted) were added to the file created by the RHC SampleSelection program.

UNIV21 44 9 9 86 9UNIV62 77 15 18 105 9UNIV61 66 13 13 81 9UNIV82 66 13 14 94 9UNIV52 38 8 9 81 9UNIV11 64 13 13 104 9


(Rev. 11/2005) Page 4-83

UNIV66 68 14 14 96 9UNIV55 71 14 15 107 9UNIV90 72 14 16 95 9UNIV86 75 15 17 101 9

After opening the two text files, the program will return to the input screen. Click onCONTINUE to resume processing. The user will need to specify the column(s) containing theinput data and the total number of columns in the sample data file using the following screen:



The sample data and primary unit information must be stored in two tables within the sameAccess database. Select the name of the database containing the input tables in the precedingOpen Data File step. This database must have the standard Access extension (.MDB). Thename of the database for this illustration is C:\TEMP\VARIABLE.MDB. The user may double-click on the VARIABLE database name or single-click on it and the Open button.

Sample Data Table

For this illustration, the data will be stored in table DATARHC2 within Access databaseC:\TEMP\VARIABLE.MDB. The first 15 rows of Access table DATARHC2 are shown next.


Page 4-84 (Rev. 11/2005)

Primary Units Table

For this illustration, there are 10 primary units (universities). The modified table (PRIMRHC2)created by the RHC Sample Selection program (the primary unit file) is shown below. Thesample sizes in the Secondary-Sample field were added to the file created by the RHC SampleSelection program.

Immediately after specifying the Access database, the user will be asked to select the name of thetables within the selected database using the form shown next. To specify the fields for theprimary unit table, click on the down arrow under Select the table. After selecting a table fromthe drop-down list (PRIMRHC2 for this illustration), click on Click here to see field names. To select the field names, click on the field name for the field containing the primary unit


(Rev. 11/2005) Page 4-85

identification (“Primary-Unit-ID” in this illustration) and click on the box labeled “Primary unitID.” The field name will appear in this box. Repeat this procedure for the remaining five fieldsin this table.

To select the field name(s) for the sample data table, first select the table (DATARHC2 in thisillustration) and click on the field name for the field containing the first piece of information inthe input file (“Difference” in this illustration) and then click on the box labeled “DifferenceValues.” The field name will then appear in this box. Repeat this procedure for any remainingfield names in this table (there are none for this illustration). When all the field names have beenspecified, click on OK. The program will return to the input screen. NOTE: When the userreturns to the input screen, the data file format (Difference Values for this illustration) will beselected, based on responses within the following Access Table and Field Names form. Whenusing an Access input file, the user cannot change this data file format option after returning tothe input screen.


Page 4-86 (Rev. 11/2005)



The sample data and primary unit information can be stored in Excel spreadsheets within thesame Excel file. Select the name of the spreadsheet containing the sample data and primary unit


(Rev. 11/2005) Page 4-87

information in the preceding Open Data File step. This file must have the standard Excelextension (.XLS). For this illustration, Excel file DATARHC2.XLS will be used.


The first 15 rows of the spreadsheet containing the sample data (named Data) are shown below. This particular file contains line numbers (1, 2, 3, . . .) in column A. These line numbers areoptional.

Primary Units Spreadsheet

When using the Excel option, the primary unit information must be contained in anotherspreadsheet within the Excel file (DATARHC2.XLS in this illustration). This spreadsheet(Primary) is shown next.


Page 4-88 (Rev. 11/2005)




(Rev. 11/2005) Page 4-89

Summary Screens

When the program has finished reading the input data files and the user clicks on CONTINUEin the initial screen, the summary screen will appear. The user can view the summary results foreach primary unit by clicking on Next P.U. and Previous P.U.. To see the summary for allprimary units combined, click on OVERALL.

When the user clicks on OK in the above screen, the screen shown below containing the firstline of the primary unit file will appear. Each line of this file can be reviewed by clicking onNext P.U. or Previous P.U.. To see a summary for the primary units combined, click onOVERALL. To continue with the appraisal, click on OK.


Page 4-90 (Rev. 11/2005)

Output Options





Program Output

If the complete option was selected by the user, the following information will be displayed forthe examined, adjusted, and difference sections of the output. The output from this programconsists of the following. The precision information is given at the two-sided 80%, 90%, and95% confidence levels.

P.U. NBR For conserving space, the sample primary unit from each group isassigned the group number. For example, the primary unit sampledfrom group #1 is assigned the number 1 for identification purposes.

PRIMARYUNIT ID

The description assigned by the user to each primary unit.

SECONDARYUNIVERSE

The number of secondary units in the population of a particular primaryunit.


(Rev. 11/2005) Page 4-91

PRIMARY UNIT SIZE

The size value selected by the user for assigning a weight to eachprimary unit. The weight factor could be, for example, number ofemployees, number of transactions, or square footage.

GROUP SIZE The summation of the size values for all the primary units in theparticular group.

UNITSIN GROUP

The number of primary units that were placed randomly in a particulargroup.

For each primary unit that is sampled, the following information is displayed:

SAMPLE SIZE The number of sampled secondary units contained in this particularsampled primary unit.

SAMPLE MEAN The average value for the appraised second-stage sample items. It isobtained by summing the items in the sample and dividing the result bythe number of sample items.

SECONDARYUNIVERSE

The number of secondary units in the universe for this particularsampled primary unit. This value was originally supplied by the user inthe file containing the Primary Unit information.

SIZES RATIO The ratio of the size of the group containing this particular primary unitto the size of the primary unit itself.

POINTESTIMATE

The estimate of the universe total for the group of primary unitscontaining this particular primary unit. For example, suppose theprimary units are split into 10 random groups, each containing 5primary units. Suppose further that the primary unit under discussionlies in group #8. The point estimate refers to the estimate of theuniverse total of the five primary units in group #8. This would berepeated for the remaining sampled primary units. The point estimateswould then be totaled to obtain an estimate of the universe total.

WITHINVARIANCE

In the derivation of the standard error, the contribution of the variabilityof the secondary units.

BETWEENVARIANCE

In the derivation of the standard error, the contribution of the variabilityof the primary units.


Page 4-92 (Rev. 11/2005)

TOTALVARIANCE

The sum of WITHIN VARIANCE and BETWEEN VARIANCE. Thesquare root of this value is the STANDARD ERROR.

The results of the above information are then used in the final overall projection. The followinginformation appears:



PRIMARYUNITS INPOPULATION


POINTESTIMATE OFPOPULATIONTOTAL

A single estimate for a universe value based on the summation of thepoint estimates for each group.

STANDARDERROR


CONFIDENCELEVEL

The user’s level of confidence (80%, 90%, or 95%) that the actualpopulation total will fall within the corresponding confidence interval.



PRECISIONAMOUNT

A measurement of the closeness of the sample estimate and thecorresponding population value. For a 90% confidence interval, theuser would be 90% confident that the estimated population total(POINT ESTIMATE OF POPULATION TOTAL) would be within thisamount of the actual value. The precision amount is calculated bymultiplying the standard error by the appropriate “Z” value (Z-VALUEUSED).


(Rev. 11/2005) Page 4-93

PRECISIONPERCENT




Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTRHC2.TXT, shown below. The printer output is identical.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 RHC TWO-STAGE VARIABLE APPRAISAL Time: 9:48 AUDIT/REVIEW: RHC - 2 Stage

DATA FILE USED: C:\temp\DATARHC2.TXT

PRIMARY SAMPLE =DIFFERENCE== NUMBER OF UNIT SIZE SAMPLE TOTAL NONZERO ITEMS ======= ====== ============= ============= 1 9 47.00 8 2 15 74.00 11 3 13 61.00 9 4 13 66.00 12 5 8 30.00 7 6 13 66.00 12 7 14 78.00 13 8 14 75.00 12 9 14 67.00 11 10 15 59.00 10

TOTALS 128 623.00 105

PRIMARY UNIT FILE USED: C:\temp\PRIMRHC2.txt

UNITSP.U. SECONDARY PRIMARY INNBR PRIMARY UNIT ID UNIVERSE UNIT SIZE GROUP SIZE GROUP==== ========================= ============= ============= ============= ===== 1 UNIV21 44 9 86 9 2 UNIV62 77 18 105 9 3 UNIV61 66 13 81 9 4 UNIV82 66 14 94 9 5 UNIV52 38 9 81 9 6 UNIV11 64 13 104 9 7 UNIV66 68 14 96 9 8 UNIV55 71 15 107 9 9 UNIV90 72 16 95 9 10 UNIV86 75 17 101 9

TOTALS 641 138 950 90


Page 4-94 (Rev. 11/2005)

DATA FILE USED: C:\temp\DATARHC2.txt

--- POINT ESTIMATES ---

===DIFFERENCE=== P.U. SAMPLE SECONDARY SIZES POINT NBR SIZE SAMPLE MEAN UNIVERSE RATIO ESTIMATE ==== ====== ============= ============= ======= ============= 1 9 5.22 44 9.556 2,196 2 15 4.93 77 5.833 2,216 3 13 4.69 66 6.231 1,930 4 13 5.08 66 6.714 2,250 5 8 3.75 38 9.000 1,283 6 13 5.08 64 8.000 2,599 7 14 5.57 68 6.857 2,598 8 14 5.36 71 7.133 2,713 9 14 4.79 72 5.938 2,046 10 15 3.93 75 5.941 1,753

TOTALS: 128 641 21,582


P.U. WITHIN BETWEEN TOTAL NBR VARIANCE VARIANCE VARIANCE 1 17,077 63,826 80,904 2 33,277 25,686 58,963 3 24,976 9,264 34,241 4 20,916 13,034 33,950 5 7,970 360,272 368,242 6 29,279 50,534 79,813 7 33,401 169,881 203,282 8 30,013 69,905 99,917 9 25,389 12,468 37,856 10 28,382 272,812 301,194

TOTALS: 250,680 1,047,683 1,298,362

DATA FILE USED: C:\temp\DATARHC2.txt


POINT ESTIMATE OF POPULATION TOTAL: 21,582

STANDARD ERROR 1,139


(Rev. 11/2005) Page 4-95






Page 4-96 (Rev. 11/2005)

Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedis a text file or printer. The screen below is the summary of the difference values for the firstprimary unit in this illustration. If the user created a sample data file containing two values persample item (e.g., examined and audited values), the summary for each amount (examined,audited, difference) can be obtained by clicking on Additional Summary Info at the bottom ofthis form. For this illustration, the Additional Summary Info button is not visible since onlythe difference values were used in the data file. To obtain the results for the second primary unit,click on Next Primary Unit. The user can click on Next Primary Unit and Previous PrimaryUnit to review the results for the individual strata. To obtain the overall results, click onOVERALL. The resulting summary screen immediately follows the summary for the firstprimary unit.


(Rev. 11/2005) Page 4-97


VARIABLE - RHC THREE STAGE RAT-STATS User’s Guide

Page 4-98 (Rev. 11/2005)

RHC THREE STAGE

Purpose

This program performs a three-stage variable appraisal using the Rao-Hartley-Cochran (RHC)methodology. This multistage procedure determines a confidence interval that uses approximate probability proportional to size (pps) sampling whereby the relative sizes of the sampling unitsare considered when selecting primary and secondary units to include in the sample. This is avariable sampling procedure allowing the user to obtain one or more quantitative pieces ofinformation about an event or item. The user has the option of obtaining and appraising fromone numeric piece of information per sample item (e.g., Examined amount) to as many as threepieces of information per sample item (i.e., Examined, Audited, and Difference amounts).

Input Screen

NOTE: The Specify Input Information button only becomes visible when the number ofprimary units in the sample have been entered.

RAT-STATS User’s Guide VARIABLE - RHC THREE STAGE

(Rev. 11/2005) Page 4-99



To enter input information, click on Specify Input Information. The form at the bottom of thepage will appear. The sample data and primary unit/secondary unit (P.U./S.U.) file format canconsist of two text files, two tables within the same Access database, or two spreadsheets withinthe same Excel file. After selecting the desired format for the data file, click on Open Data File. Note: If the Access File (.MDB) option is selected, this button will change to Open Accessdatabase and if the Excel file (.XLS) option is selected, this button will change to Open Excelfile. Using the standard Windows “Open” file screen, locate the sample data file and doubleclick on it.

If the Text files option is being used, the Open P.U./S.U. File button will appear in the formbelow after the sample data file has been opened. The P.U./S.U. file is a slightly modifiedversion of the file created by the RHC Sample Selection program. Click on the Open P.U./S.U.File button and, using the standard Windows “Open” file screen, locate the P.U./S.U. file anddouble-click on it.



Page 4-100 (Rev. 11/2005)


Prior to executing this program, the user must create a sample data file that contains certainidentifying data and one or two pieces of information for each sample unit selected. Each dataline consists of a line number for that sampling unit followed by the first piece of information (anumeric value) the user wants to appraise (i.e., Examined, Audited, or Difference value). If twoor more pieces of information will be appraised and the examined amount is one of the values,then the examined amount must be the first piece of data entered for each sampling unit. If onlythe audited and difference amounts are being appraised, then the audited amount must be the firstpiece of data entered. The second piece of information may be the numeric difference between


(Rev. 11/2005) Page 4-101

the examined value and the amount accepted by the user or the audited amount if the examinedamount was the first piece of data entered.


7483 289.99 43.00

Explanation:



43.00 - If two pieces of information are gathered for each sampling unit, then this numberis the audited or difference amount determined by the user. For example, if theuser had determined that of $289.99 claimed by a vendor, only $246.99 wasactually owed, then the difference amount entered would be $43.00.


Format of P.U./S.U. Information

When the RHC Sample Selection program is used to select the primary and secondary units, afile is created when selecting the primary units and additional files are created for the selectionof the secondary units within each sampled primary unit. These files contain information on theclustering of the primary (or secondary) units in the universe as well as the primary (orsecondary) units selected. These files may be text file files, tables within an Access database, orExcel spreadsheets. These files must be combined into a single file as illustrated below. It isthis combined file that must be opened when clicking on the Open P.U./S.U. File button.

The information in this combined file must be aligned in a set format. In the P.U./S.U. file, theprimary units followed by the secondary units are in the same order as in the sample data filepreviously discussed. The primary universe information is entered first, followed by the data forthe secondary universes.

Regardless of the software used to create the P.U./S.U. file, the format should be as follows:


Page 4-102 (Rev. 11/2005)

PRIMARY #1 8 3 6,000 18,000 6SECONDARY #1 350 20 1,000 1,400 10

Explanation:

PRIMARY #1 - This is a brief description of the sampled primary unit. The descriptionshould be no longer than 30 characters in length.

8 - This is the quantity of secondary units in this primary unit. The user mayinclude commas in the value.

3 - This is the number of secondary units sampled from this primary unit forthis appraisal. Commas may be used in the value.

6,000 - This is the size factor assigned by the user to the primary unit (e.g., squarefootage, number of transactions, etc.). Commas and decimal points maybe included in the values.

18,000 - This is the size factor for the group from which the primary unit wasselected. Commas and decimal points may be included in the values.

6 - This is the number of primary units in the group from which this primaryunit was selected. Commas may be included in the value.

SECONDARY #1 -This is a brief description of the sampled secondary unit. A maximum of 30characters, including spaces, will be accepted.

350 - This is the quantity of third-stage units in the universe for this secondaryunit. The entry may contain commas and a decimal point.

20 - This is the quantity of third-stage units in the sample for this secondaryunit. The entry may contain commas and a decimal point.

1,000 - This is the secondary size factor used in weighting the secondary unit(e.g., square footage, number of transactions, etc.). Commas and adecimal point may be incorporated in the number.

1,400 - This is the size factor for the secondary group that this secondary item wassampled from. Commas and a decimal point may be used in the value.

10 - This is the number of secondary units in this group from which thissecondary unit was selected. Commas may be used in the value.


(Rev. 11/2005) Page 4-103

The user must enter the information for primary units and secondary units on a separate line. Forease of editing, enough spaces or tabs should be placed between values so that the sample valuesalign vertically. The program assumes one or more spaces or tabs as the only delimiters betweenpieces of data.


The data files may be stored in a text file format. There are several ways the users may createthis file. The easiest approach would be with a word processing package (e.g., Word) or a printfile created with a spreadsheet package (e.g., Excel).

Sample Data File

As an illustration, four primary units (REGIONS 3, 7, 8, and 9) were selected from a total of 12using the RHC Sample Selection program. Ten universities were also selected from each of thefour regions using the RHC Sample Selection program. The file on the next page(C:\TEMP\DATARHC3.TXT) was obtained by sampling the 10 selected universities within eachregion and recording the difference values. (NOTE: Only a portion of the data for REGION 3and REGION 9 is shown.)

1 8 μ Primary unit 1 (REGION 3), secondary unit 1 (UNIV20)2 03 64 65 06 137 18 7 μ Primary unit 1 (REGION 3), secondary unit 2 (UNIV38)9 210 1311 1312 413 614 015 1516 1217 918 019 1320 10 . . .92 11 μ Primary unit 1 (REGION 3), secondary unit 10 (UNIV54)93 694 1095 1196 097 798 12


Page 4-104 (Rev. 11/2005)


99 9100 11 . . .319 0 μ Primary unit 4 (REGION 9), secondary unit 1 (UNIV37)320 10321 14322 0323 18324 0325 8326 20327 19328 0329 0330 3331 3332 13 μ Primary unit 4 (REGION 9), secondary unit 2 (UNIV92)333 0334 12335 12336 7337 1338 13339 2340 0341 16342 14343 17344 5345 8 . . .423 19 μ Primary unit 4 (REGION 9), secondary unit 10 (UNIV107)424 17425 13426 12427 13428 12429 11430 14431 13432 0433 1


(Rev. 11/2005) Page 4-105

P.U./S.U. File

For this illustration, there are four primary units (regions) with 10 secondary units (universities)selected within each sampled primary unit. The file (C:\TEMP\PUSURHC3.TXT) created bycombining the results of the five RHC Sample Selection programs (one for selecting the primaryunits and four for selecting the secondary units) is shown on the next page. The sample sizes(highlighted) were added to the files created by the five RHC Sample Selection programs.

REGION 3 91 10 720 3280 3UNIV20 37 7 6 73 9UNIV38 74 15 11 70 9UNIV45 73 15 11 82 9UNIV10 60 12 9 81 9UNIV87 62 12 10 59 9UNIV82 30 6 5 68 9UNIV60 54 11 9 75 9UNIV69 39 8 7 76 9UNIV34 26 5 4 60 9UNIV54 57 11 9 76 10REGION 7 102 10 960 2210 3UNIV1 56 11 10 89 10UNIV60 56 11 10 96 10UNIV59 67 13 13 94 10UNIV99 80 16 14 91 10UNIV85 67 13 13 93 10UNIV37 31 6 6 103 10UNIV34 42 8 8 106 10UNIV16 53 11 10 83 10UNIV12 66 13 13 90 11UNIV52 60 12 11 115 11REGION 8 118 10 1300 3710 3UNIV19 34 7 8 137 11UNIV104 77 15 16 122 11UNIV66 49 10 11 122 12UNIV110 38 8 9 137 12UNIV83 70 14 15 121 12UNIV14 48 10 10 147 12UNIV78 27 5 7 122 12UNIV105 27 5 6 144 12UNIV12 65 13 14 122 12UNIV112 75 15 16 126 12REGION 9 122 10 1320 2800 3UNIV37 64 13 14 147 12UNIV92 73 15 15 125 12UNIV47 71 14 15 130 12UNIV54 70 14 15 131 12UNIV97 56 11 12 138 12


Page 4-106 (Rev. 11/2005)

UNIV66 76 15 16 122 12UNIV116 50 10 10 140 12UNIV29 33 7 8 128 12UNIV18 26 5 7 132 13UNIV107 55 11 11 127 13

After opening the two text files, the program will return to the input screen. Click onCONTINUE to resume processing. The user will need to specify the column(s) containing theinput data and the total number of columns in the sample data file using the screen below.



The sample data and primary/secondary unit information must be stored in two tables within thesame Access database. Select the name of the database containing the input tables in thepreceding Open Data File step. This database must have the standard Access extension(.MDB). The name of the database for this illustration is C:\TEMP\VARIABLE.MDB. Theuser may double-click on the VARIABLE database name or single-click on the filename and theOpen button.

Sample Data Table

For this illustration, the sample data will be stored in table DATARHC3 within Access databaseC:\TEMP\VARIABLE.MDB.

The first 15 rows of Access table DATARHC3 are shown next.


(Rev. 11/2005) Page 4-107

P.U./S.U. Table

The modified table (PUSURHC3) created by combining the five passes through the RHCSample Selection program and inserting the sample size field (Units-Sample) is shown below. This table is referred to as the Table for the primary/secondary units.


Page 4-108 (Rev. 11/2005)

Immediately after specifying the Access database, the user will be asked to select the name of thetables within the selected database using the form on the next page. To specify the fields for theP.U./S.U. table, click on the down arrow under Select the table. After selecting a table from thedrop-down list (PUSURHC3 for this illustration) click on Click here to see field names. Toselect the field names, click on the field name for the field containing the unit identification(“ID” in this illustration) and click on the box labeled “Unit ID.” The field name will appear inthis box. Repeat this procedure for the remaining five fields in this table.

To select the field name(s) for the sample data file, first select the table (DATARHC3 in thisillustration) and click on the field name for the field containing the first piece of information inthe input file (“Difference” in this illustration) and then click on the box labeled “DifferenceValues.” The field name will then appear in this box. Repeat this procedure for any remainingvariables of interest in this table (the only variable of interest in this illustration is the differenceamount). When all the field names have been specified, click on OK. The program will thenreturn to the input screen. NOTE: When the user returns to the input screen, the data file format(Difference Values for this illustration) will be selected, based on responses within the followingAccess Table and Field Names form. When using an Access input file, the user cannot changethis data file format option after returning to the input screen.


(Rev. 11/2005) Page 4-109



Page 4-110 (Rev. 11/2005)


The sample data and P.U./S.U. information can be stored in Excel spreadsheets within the sameExcel file. Select the name of the spreadsheet containing the sample data and P.U./S.U.information in the preceding Open Data File step. This file must have the standard Excelextension (.XLS). For this illustration, Excel file DATARHC3.XLS will be used.


The first 15 rows of this spreadsheet (named Data) are shown below. This particular filecontains line numbers (1, 2, 3, . . .) in column A. These line numbers are optional.

P.U./S.U. Spreadsheet

When using the Excel option, the P.U./S.U. information must be contained in anotherspreadsheet within the Excel file (DATARHC3.XLS in this illustration). A portion of thisspreadsheet (named P.U.-S.U.) is shown on the next page.


(Rev. 11/2005) Page 4-111



Page 4-112 (Rev. 11/2005)



(Rev. 11/2005) Page 4-113

Output Options


If the user selects a text file for output, the program will prompt for a file name. The standardWindows “Save” file screen will appear. The user should type in the file name in the designatedbox and click on Save. The output file will be saved with a “TXT” extension.



Program Output

The following information will be displayed for the examined, adjusted, and difference sectionsof the output. The precision information is given at the two-sided 80%, 90%, and 95%confidence levels.

The output comes in three sections. The first section provides summary information from thesample data and universe files identified by the user. The next section displays statistics for eachof the secondary and primary units. In addition, the results of variance calculations aredisplayed. The third section displays the overall projections of the appraisal.

PRIMARY/SECONDARYIDENTIFICATION

The description information obtained from the file of universe dataidentified by the user.

THIRD-STAGEUNIVERSE

The number of third-stage units that make up the population within therespective secondary unit.


Page 4-114 (Rev. 11/2005)

SAMPLE SIZE The number of third-stage items sampled in the secondary unit.

SAMPLE VALUE The summation of the values entered for each of the third-stagesample items in the secondary unit.

NONZERO COUNT The number of third-stage sample items that had a value other thanzero.

SAMPLE MEAN The average value for the third-stage items appraised. It is obtainedby summing the items in the sample and dividing the result by thenumber of items in the sample.

SIZES RATIO The ratio of the size of the group containing this particular secondaryunit to the size of the secondary unit itself. The size factors areobtained from the universe file indicated by the user.

POINT ESTIMATE The estimate of the universe total for the group of secondary unitscontaining this particular secondary unit. For example, suppose thesecondary units are split into 10 random groups, each containing 5secondary units. Suppose further that the sampled secondary unit liesin group #8. The point estimate refers to the estimate of the universetotal of the five secondary units in group #8. This would be repeatedfor the remaining sampled secondary units.


WITHINVARIANCE

For each sampled primary unit, the contribution of the third-stagevariation.

BETWEENVARIANCE

For each sampled primary unit, the contribution of the second-stagevariation.

TOTALVARIANCE

For each sampled primary unit, the sum of WITHIN VARIANCE andBETWEEN VARIANCE. This value represents the total variationobtained by applying a RHC two-stage procedure to the sampledprimary unit.


(Rev. 11/2005) Page 4-115


STAGE 1 In the derivation of the standard error, the contribution of the first-stage (primary) units.

STAGES 2 AND 3 In the derivation of the standard error, the contribution of the second-stage (secondary) and third-stage units.

TOTALVARIANCE

The sum of the values for STAGE 1 and STAGES 2 AND 3. Thesquare root of this value is the STANDARD ERROR.

OVERALL SECTION: The results of the above information are then used in the final overallprojection. The following information appears:

PRIMARY UNITSSAMPLED

The quantity of primary units selected in this sample.

PRIMARY UNITSNOT SAMPLED


TOTAL PRIMARYUNITS


OVERALL POINTESTIMATE

A single estimate for a universe value based on each primary unitpoint estimate multiplied by (A/B) where A is the size of the groupcontaining the primary unit and B is the size of the primary unit. Theproducts are summed over all primary units.



CONFIDENCELEVEL

This user’s level of confidence (80%, 90%, or 95%) that the actualpopulation total will fall within the corresponding confidence interval.




Page 4-116 (Rev. 11/2005)

PRECISIONAMOUNT

A measurement of the closeness of the sample estimate and thecorresponding population value. For a 90% confidence interval, theuser would be 90% confident that the estimated population total(OVERALL POINT ESTIMATE) would be within this amount of theactual value. The precision amount is calculated by multiplying thestandard error by the appropriate factor (“Z” value).

PRECISIONPERCENT




Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTRHC3.TXT, shown below. The printer output is identical.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/28/2004 RHC THREE-STAGE VARIABLE APPRAISAL Time: 10:45 AUDIT/REVIEW: RHC 3-Stage

DATA FILE USED: C:\temp\DATARHC3.TXT PRIMARY/SECONDARY UNIVERSE FILE USED: C:\temp\PUSURHC3.TXT

*****D I F F E R E N C E***** **** SAMPLED UNITS **** Third-stage SAMPLE SAMPLE NONZEROPRIMARY / SECONDARY IDENTIFICATION UNIVERSE SIZE VALUE COUNT================================== =========== ====== ============ ========REGION3 UNIV20 37 7 34.00 5 UNIV38 74 15 116.00 12 UNIV45 73 15 148.00 15 UNIV10 60 12 103.00 11 UNIV87 62 12 97.00 10 UNIV82 30 6 28.00 4 UNIV60 54 11 44.00 9 UNIV69 39 8 84.00 8 UNIV34 26 5 40.00 4 UNIV54 57 11 92.00 10 Total 512 102 88


(Rev. 11/2005) Page 4-117

REGION7 UNIV1 56 11 115.00 9 UNIV60 56 11 106.00 8 UNIV59 67 13 92.00 13 UNIV99 80 16 111.00 12 UNIV85 67 13 139.00 10 UNIV37 31 6 23.00 3 UNIV34 42 8 45.00 7 UNIV16 53 11 79.00 7 UNIV12 66 13 102.00 9 UNIV52 60 12 123.00 11 Total 578 114 89



TOTALS 2,174 433 3,385.00 352

**** SAMPLED UNITS **** SIZES POINT PRIMARY / SECONDARY IDENTIFICATION SAMPLE MEAN RATIO ESTIMATE ================================== =========== =========== =============REGION3 UNIV20 4.86 12.1667 2,187 UNIV38 7.73 6.3636 3,642 UNIV45 9.87 7.4545 5,369 UNIV10 8.58 9.0000 4,635 UNIV87 8.08 5.9000 2,957 UNIV82 4.67 13.6000 1,904 UNIV60 4.00 8.3333 1,800 UNIV69 10.50 10.8571 4,446 UNIV34 8.00 15.0000 3,120 UNIV54 8.36 8.4444 4,026

TOTAL 34,085


Page 4-118 (Rev. 11/2005)


TOTAL 40,896


TOTAL 41,665


TOTAL 50,916


**** SAMPLED UNITS **** WITHIN BETWEEN TOTALPRIMARY UNIT IDENTIFICATION VARIANCE VARIANCE VARIANCE============================== ============== ============== ==============REGION3 434,942 9,869,853 10,304,795REGION7 892,072 14,293,056 15,185,128REGION8 499,759 14,612,210 15,111,969REGION9 1,134,688 20,534,696 21,669,384


(Rev. 11/2005) Page 4-119




1,148,658,590 170,994,089 1,319,652,679

--- SUMMARY OF APPRAISAL RESULTS ---PRIMARY UNITS SAMPLED 4PRIMARY UNITS NOT SAMPLED 8TOTAL PRIMARY UNITS 12OVERALL POINT ESTIMATE 476,331OVERALL STANDARD ERROR 36,327




Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedis a text file or printer. The screen below is the summary of the difference values for the firstprimary unit in this illustration. If the user created a sample data file containing two values persample item (e.g., examined and audited values), the summary for each amount (examined,audited, difference) can be obtained by clicking on Additional Summary Info at the bottom ofthis form. For this illustration, the Additional Summary Info button is not visible since onlythe difference values were used in the data file.


Page 4-120 (Rev. 11/2005)

To view summary information for the next sampled secondary unit within the selected primaryunit, click on Next in the bottom Secondary Unit box. By clicking on Next and Previous, theuser can navigate through the secondary units for the selected primary unit. By clicking onCombined, the screen will contain the summary for the selected primary unit. Similarly, byclicking on Next and Previous in the Primary Unit box, the user can navigate through thesampled primary units. For each selected primary unit, the user can then switch to theSecondary Unit box and examine the results for each secondary unit within this primary unit. Finally, by clicking on Overall in the Primary Unit box, the summary appraisal results for theentire analysis are shown. These output screens are shown on the next two pages.


(Rev. 11/2005) Page 4-121

NOTE Example is for illustrative purposes only. The sample sizes maynot conform to the organization’s minimum sample size standards.

VARIABLE - STRATIFIED CLUSTER RAT-STATS User’s Guide

Page 4-122 (Rev. 11/2005)

STRATIFIED CLUSTER

Purpose

This program performs a stratified cluster appraisal of a previously created data file based oninformation gathered from all items in sampled clusters. The user would have initially stratifiedthe clusters (e.g., universities) into two or more categories (e.g., public and private universities). Within each stratum, the user would randomly select clusters. Then all items (e.g., grants)within the cluster would be reviewed.

Input Screen



RAT-STATS User’s Guide VARIABLE - STRATIFIED CLUSTER

(Rev. 11/2005) Page 4-123

Number of strata

After entering the name of the audit/review, the user must enter the number of strata. Themaximum number of strata is 100.

Specify input information

After entering the above information, click on Specify Input Information. The followingscreen will appear. The input file format can be a text file, a table within an Access database, oran Excel spreadsheet. After selecting the desired format, click on Open Data File. Using thestandard Windows “Open” file screen, locate the input file and double-click on it.

When the file has been opened, the full input screen (shown next) will appear.


Page 4-124 (Rev. 11/2005)


Prior to executing this program the user must create a sample data file that contains certainidentifying data and sample information for each cluster selected. The identifying data aredescriptive identifiers of the stratum and each cluster (maximum of 25 characters for eachidentifier). For each stratum the user enters the number of clusters (e.g., universities) that makesup the universe for the stratum. For each cluster sampled the user enters the number of items(e.g., grants) in the cluster and the total value for all items in the cluster of the quantitativecharacteristic being measured (e.g., unauthorized travel costs).

For each stratum the user needs to use the following format:


(Rev. 11/2005) Page 4-125

PRIVATE UNIVERSITIES 415 25Explanation:

PRIVATE -UNIVERSITIES

This is a description of the clusters in the stratum. It must be least onecharacter in length. It may be longer than 25 characters in length;however, only the first 25 characters will be stored for output.

415 - This is the number of clusters in the universe for the stratum.

25 - This is the number of clusters in the sample for the stratum.

The data values may contain commas and dollar signs ($). The program assumes one or morespaces or tabs as the only delimiters between pieces of data.

Each stratum data line is followed by one or more lines containing data for each cluster sampled. The format for each cluster is as follows:

UNIV1 8 96Explanation:

UNIV1 - This is a description of the sampled cluster. It must be at least onecharacter in length. It may be longer than 25 characters in length;however, only the first 25 characters will be stored for output.

8 - This is the number of items reviewed within the cluster. The quantitymust be separated from the cluster description by one or more spaces. Commas are allowed for this value.

96 - This is the total value for the quantitative characteristic being measuredfor the items in the cluster. The value must be separated by one or morespaces from the number of items reviewed in the cluster. The valuemay contain commas, a decimal point, or a dollar sign. If the value isnegative, it must be preceded by a minus sign (-).


The sample data containing the above information may be stored in a text file format. There areseveral ways the user may create this file. The easiest approach would be with a wordprocessing package (e.g., Word) or a print file created with a spreadsheet package (e.g., Excel).


Page 4-126 (Rev. 11/2005)

NOTE: Example is for illustrativepurposes only. The sample sizes may notconform to the organization’s minimumsample size standards.

As an illustration, consider two strata: Stratum 1: state-supported universities (N1 = 415universities in the population) and Stratum 2: private universities (N2 = 168 universities in thepopulation). Within each stratum, a single-stage cluster sample was obtained with n1 = 25universities selected from Stratum 1 and n2 = 10 universities from Stratum 2. For each of thesampled universities, all health-related grants would be audited to determine the amount of costsimproperly charged to these grants. The sample data file (C:\TEMP\DATASTRCLUS.TXT) isshown below.

STATE UNIVERSITIES 415 25UNIV1 8 96UNIV2 12 121UNIV3 4 42UNIV4 5 65UNIV5 6 52UNIV6 6 40UNIV7 7 75UNIV8 5 65UNIV9 8 45UNIV10 3 50UNIV11 2 85UNIV12 6 43UNIV13 5 54UNIV14 10 49UNIV15 9 53UNIV16 3 50UNIV17 6 32UNIV18 5 22UNIV19 5 45UNIV20 4 37UNIV21 6 51UNIV22 8 30UNIV23 7 39UNIV24 3 47UNIV25 8 41PRIVATE UNIVERSITIES 168 10UNIV1 2 18UNIV2 5 52UNIV3 7 68UNIV4 4 36UNIV5 3 45UNIV6 8 96UNIV7 6 64UNIV8 10 115UNIV9 3 41UNIV10 1 12


(Rev. 11/2005) Page 4-127


The sample data may be stored in a table within an Access database. Select the name of thedatabase containing the input table in the preceding Open Data File step. This database musthave the standard Access extension (.MDB). The name of the database for this illustration isC:\TEMP\VARIABLE.MDB. The user may double-click on the VARIABLE database name orsingle-click on it and the Open button on the standard Windows “Open” file screen.

For this illustration, the data will be stored in table DATASTRCLUS within Access databaseC:\TEMP\VARIABLE.MDB. The first seven rows of Access table DATASTRCLUS are shownbelow. The final 15 rows immediately follow.

Once the data file is open, the program returns to the input screen. Click on CONTINUE toresume processing. The user will be asked to select the name of the table within the selected


Page 4-128 (Rev. 11/2005)


database using the following form. Click on the down arrow under Select a table. Afterselecting a table from the drop-down list (DATASTRCLUS for this illustration) and clicking onClick here to see field names, the field names will appear.

To select the field names, click on the field name for the field containing the primary andsecondary unit IDs (“Field1” in this illustration) and then click on the top right-hand box. Thefield name will then appear in this box. Repeat this procedure for the remaining two field namesin this table (“Field2” and “Field3” in this illustration). When all the field names have beenspecified, click on OK. The program will continue processing.


(Rev. 11/2005) Page 4-129

NOTE: Example is forillustrative purposes only. The sample sizes may notconform to the organization’sminimum sample sizestandards.


The input file can be stored in an Excel spreadsheet containing three columns. Select the nameof the spreadsheet containing the input table in the preceding Open Data File step. This filemust have the standard Excel extension (.XLS). For this illustration, Excel fileDATASTRCLUS.XLS will be used. The first seven rows of this spreadsheet are shown below. The final 15 rows immediately follow.

When using the Excel option, the following screen will be displayed.


Page 4-130 (Rev. 11/2005)



Output Options






(Rev. 11/2005) Page 4-131

Program Output

Whether the user selects the complete or summary option for output, the following informationwill be displayed for each stratum:

STRATUMIDENTIFICATION

The descriptions entered in the sample data file.

SAMPLEUNIVERSE

The universe size of clusters for each stratum. The numbers wereobtained from the data file.

SAMPLE SIZE The count of clusters in the data file for each stratum. It represents thenumber of clusters reviewed in each stratum.

If the user selects the complete option, the following information will be displayed:

CLUSTERIDENTIFICATION

The descriptions entered in the data file.

SAMPLEUNIVERSE

The universe size for each cluster in a stratum. The numbers wereobtained from the data file.

SAMPLE SIZE The number of items actually reviewed. Since this is a cluster sample,the universe and sample sizes are the same within each cluster. Thenumber of items may vary from cluster to cluster, but all items in allsampled clusters are to be reviewed.

SAMPLED VALUE The summation of values entered in the data file for the quantitativecharacteristic being measured. It represents the summation for eachcluster.

The final two pieces of information for each stratum (sampled value and point estimate) willappear on different lines depending on the output option selected (i.e., complete or summary). Ifa complete option is selected, then after displaying the information for each sampled cluster in astratum, a stratum totals line will appear containing these two values. If, however, the summaryoption was selected, then these two values will appear on the same line as the stratumidentification.


Page 4-132 (Rev. 11/2005)

SAMPLED VALUE The summation of values entered in the data file for the quantitativecharacteristic being measured. In this line it represents the summationfor the stratum.

POINT ESTIMATE A single estimate for the stratum of the universe value of thecharacteristic being measured. It is obtained by dividing thesummation of stratum sampled values by the number of clusterssampled and multiplying the result by the number of clusters in thestratum universe.

Whether the complete or summary option is selected, the following output will be displayed afterall the information for the strata are shown:

STRATA TOTALS The total number of clusters in the universe and the actual number ofclusters sampled.

CLUSTER UNITTOTALS

Information for all clusters sampled including the total number ofitems reviewed and the value of those items for the characteristicbeing measured.

OVERALL POINTESTIMATE

A single estimate of the overall value of the characteristic beingmeasured. It is calculated by the summation of the point estimates forall strata.


A measurement of the variance of the overall point estimate. It is thisvalue that is used in determining the width of the confidence intervals.

CONFIDENCELEVEL

The user’s level of confidence that the actual value of the measuredcharacteristic will fall within the range from the lower to the upperlimits (confidence interval).

LOWER LIMIT The lower bound of the confidence interval. It is calculated bysubtracting the precision amount from the point estimate.

UPPER LIMIT The upper bound of the confidence interval. It is calculated by addingthe precision amount to the point estimate.


(Rev. 11/2005) Page 4-133

PRECISIONAMOUNT

A measurement of the closeness of the sample estimate of the universetotal and the corresponding unknown universe value. The precisionamount is calculated by multiplying the standard error by theappropriate factor (“Z” value) corresponding to the desired confidencelevel. For the examined (reviewed) appraisal, the universe total maybe known and should be reviewed by the user to see if, in fact, theactual value does fall within the confidence interval.

PRECISIONPERCENT




Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTSTRCLUS.TXT, shown next. The printer output is identical.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 STRATIFIED CLUSTER VARIABLE APPRAISAL Time: 14:36 AUDIT/REVIEW: Variable - Stratified Cluster

DATA FILE USED: C:\TEMP\DATASTRCLUS.TXT

STRATUM IDENTIFICATION SAMPLE SAMPLE CLUSTER IDENTIFICATION UNIVERSE SIZE SAMPLED VALUE POINT ESTIMATE=========================== =========== ====== ============= ==============STATE UNIVERSITIES 415 25 UNIV1 8 8 96.00 UNIV2 12 12 121.00 UNIV3 4 4 42.00 UNIV4 5 5 65.00 UNIV5 6 6 52.00 UNIV6 6 6 40.00 UNIV7 7 7 75.00 UNIV8 5 5 65.00 UNIV9 8 8 45.00 UNIV10 3 3 50.00 UNIV11 2 2 85.00 UNIV12 6 6 43.00 UNIV13 5 5 54.00 UNIV14 10 10 49.00 UNIV15 9 9 53.00 UNIV16 3 3 50.00 UNIV17 6 6 32.00 UNIV18 5 5 22.00 UNIV19 5 5 45.00 UNIV20 4 4 37.00 UNIV21 6 6 51.00 UNIV22 8 8 30.00 UNIV23 7 7 39.00


Page 4-134 (Rev. 11/2005)


UNIV24 3 3 47.00 UNIV25 8 8 41.00 STRATUM TOTALS 151 151 1,329.00 22,061

PRIVATE UNIVERSITIES 168 10 UNIV1 2 2 18.00

UNIV2 5 5 52.00 UNIV3 7 7 68.00 UNIV4 4 4 36.00 UNIV5 3 3 45.00 UNIV6 8 8 96.00 UNIV7 6 6 64.00 UNIV8 10 10 115.00 UNIV9 3 3 41.00 UNIV10 1 1 12.00 STRATUM TOTALS 49 49 547.00 9,190

STRATUM IDENTIFICATION SAMPLE SAMPLE CLUSTER IDENTIFICATION UNIVERSE SIZE SAMPLED VALUE POINT ESTIMATE=========================== =========== ====== ============= ==============

STRATA TOTALS 583 35CLUSTER UNIT TOTALS 200 200 1,876.00OVERALL POINT ESTIMATE 31,251OVERALL STANDARD ERROR 2,418


Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedincludes a text file or printer. The screen immediately following is the summary of thedifference values for the first stratum in this illustration. To obtain the results for the secondstratum, click on Next Stratum. The user can click on Next Stratum and Previous Stratum toreview the results for the individual strata. To obtain the overall results, click on OVERALL. The resulting summary screen immediately follows the summary for the first stratum.


(Rev. 11/2005) Page 4-135

VARIABLE - STRATIFIED MULTISTAGE RAT-STATS User’s Guide

Page 4-136 (Rev. 11/2005)


Purpose

This program performs a stratified multistage appraisal based on information gathered from priormultistage appraisals. The user would have initially stratified the clusters (e.g., universities) intotwo or more categories (e.g., public and private universities). Within each stratum, the userwould select a multistage sample. The results of the samples would be appraised using amultistage appraisal program. The point estimate and standard error from each of theseappraisals could be placed in a data file or entered interactively by the user. NOTE: Use thepoint estimates and standard errors for the totals, not the point estimates and standard errors forthe means.

Input Screen

RAT-STATS User’s Guide VARIABLE - STRATIFIED MULTISTAGE

(Rev. 11/2005) Page 4-137



Number of strata for this appraisal

Enter the number of strata that will be appraised. This number must be between 2 and 500.

Have you created a text data file for this appraisal?

Prior to executing this program the user could create a data file that contains the point estimateand the standard error for each of the strata. The data file containing the above information mustbe stored in a text file format. There are several ways the user may create this file. The easiestapproach would be with a text editor or word processing package (e.g., Word) or a print filecreated with a spreadsheet package (e.g., Excel).

For each stratum the user needs to use the following format:

24,077 1,277

Explanation:

24,077 - The point estimate of the total for one stratum. The program will allow commasto be included in the value.

1,277 - The standard error of the total estimate for one stratum. The program will allowcommas to be included in the value.

The results for each stratum should be on a separate line and the values separated by one or morespaces or tabs.

If Yes is selected, the standard Windows “Open” file screen will appear. The window willcontain all “.TXT” and “.DAT” files in the selected directory. To view all files in this directory,the user may change the file type to “All files.” Click on the file name, then click on Open. If afile name is entered, the values obtained from the file will be displayed on the screen for reviewby the user.

If No is selected at this query the following screen will appear:


Page 4-138 (Rev. 11/2005)

Input of Data Values

Entering point estimates

For the stratum identified on the screen, the user may enter a point estimate of the stratum total.

Entering standard errors

For the stratum identified on the screen, the user may enter a standard error for the estimate ofthe stratum total.

Editing the values

By clicking on Next Stratum and Previous Stratum, the user can navigate through the strata toobserve, enter, or modify the point estimates and standard errors.

At any point before clicking on the CONTINUE button, the user may change the point estimateor standard error for any of the strata. When all edits are complete, click on Click here to savethe data set. The standard Windows “Save As” screen will appear. Type the output file namealongside the File name box and click on Save.


(Rev. 11/2005) Page 4-139

Output Options





Program Output

The output will first display the data entered by the user for each of the strata. Then the outputwill display the results of the appraisal. The following will be displayed in the results portion:

POINTESTIMATE

A single estimate for all strata of the universe value of thecharacteristic being measured. It is obtained by the summation ofstrata point estimates.

STANDARDERROR

A measurement of the variation of the point estimate of the universetotal with respect to all possible point estimates for this universe andthese sample sizes.

CONFIDENCELEVEL

The user’s level of confidence that the actual value of the measuredcharacteristic will fall within the range from the lower to upper limits(confidence interval).


Page 4-140 (Rev. 11/2005)

LOWER LIMIT The lower bound of the confidence interval. It is calculated bysubtracting the precision amount from the point estimate.

UPPER LIMIT The upper bound of the confidence interval. It is calculated by addingthe precision amount to the point estimate.

PRECISIONAMOUNT

A measurement of the closeness of the sample estimate and thecorresponding population value. For a 90% confidence interval, theuser would be 90% confident that the estimated population total(OVERALL POINT ESTIMATE) would be within this amount of theactual value. The precision amount is calculated by multiplying thestandard error by the appropriate factor (“Z” value).

PRECISIONPERCENT




(Rev. 11/2005) Page 4-141



Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTSTRMULTI.TXT, shown below. The printer output is identical.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 STRATIFIED MULTISTAGE VARIABLE APPRAISAL Time: 10:09 AUDIT/REVIEW: Variable - Stratified Multistage DATA FILE: C:\TEMP\DATASTRMULTI.TXT

THE ESTIMATORS ARE BASED ON THE FOLLOWING ENTRIES: STRATUM POINT ESTIMATE STANDARD ERROR 1 24,077 1,277 2 19,182 873

= = = = = = = = = = = = = = = = = = RESULTS = = = = = = = = = = = = = = = = =

POINT ESTIMATE STANDARD ERROR

43,259 1,547



Page 4-142 (Rev. 11/2005)


Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedincludes a text file or printer. The summary screen for this illustration is shown on the nextpage. The input values are contained in the box at the bottom of the screen.

RAT-STATS User’s Guide VARIABLE - POSTSTRATIFICATION

(Rev. 11/2005) Page 4-143

POSTSTRATIFICATION

Purpose

While in many cases the user would like to develop a stratified sampling plan, it may not befeasible to do so before actually drawing the sample, or the user may not recognize the need tostratify until after the sample has been drawn and the items evaluated. In such situations poststratification may be used. Such a stratification methodology may only be used if the sizes of thestrata populations are known and each stratum sample is of sufficient size. This method,however, is less efficient statistically than a stratified sample.

Input Screen

VARIABLE - POSTSTRATIFICATION RAT-STATS User’s Guide

Page 4-144 (Rev. 11/2005)



Number of strata

After entering the name of the audit/review, the user must enter the number of strata.

Specify input file

After entering the above information, click on Specify Input File. The following screen willappear. The input file(s) format can consist of two text files, two tables within an Accessdatabase, or two spreadsheets within the same Excel file. After selecting the desired format,click on Open Data File. Using the standard Windows “Open” file screen, locate the input fileand double-click on it.

When all files have been opened, the full input screen (shown next) will appear.


(Rev. 11/2005) Page 4-145


Prior to executing this program, the user must create a sample data file that contains certainidentifying data and one or two pieces of information for each sample unit selected within eachstratum. Each data line consists of a line number for that sampling unit followed by the firstpiece of information (a numeric value) the user wants to appraise (i.e., Examined, Audited, orDifference value). If two or more pieces of information will be appraised and the examinedamount is one of the values, then the examined amount must be the first piece of data entered foreach sampling unit. If only the audited and difference amounts are being appraised, then theaudited amount must be the first piece of data entered. The second piece of information may bethe numeric difference between the examined value and the amount accepted by the user or theaudited amount if the examined amount was the first piece of data entered.


Page 4-146 (Rev. 11/2005)


7483 289.99 43.00Explanation:

7483 - This is a number assigned by the user. The user should use the sample itemnumber in this position. For ease of reference in this example, the number will bereferred to as the line number.





The data file containing the above information may be stored in a text file format. There areseveral ways the user may create this file. The easiest approach would be with a wordprocessing package (e.g., Word) or a print file created with a spreadsheet package (e.g., Excel).

Sample Data File

A portion of data set DATAPOST.TXT is shown next The data file contains 25 observations ineach stratum.


(Rev. 11/2005) Page 4-147


1 80 2 43 3 133 4 125 5 116 . Data file DATAPOST.TXT . .21 12722 10523 10224 6925 7626 35427 32828 31329 25030 261 . . .46 29547 27748 35549 31450 277

After clicking on the Open Data File button and opening the data file (e.g., DATAPOST.TXT),the Open/Read Universe File button will appear on the following screen.


Page 4-148 (Rev. 11/2005)

Universe/Sample Sizes File

To open the file containing the universe/sample sizes, click on the Open/Read Universe File button and use the standard Windows “Open” file screen to locate the universe file(e.g., UnivPost.TXT, shown below). The format of this file is:


There is one line for each stratum in the sample. Values within a line can be separated by one ormore spaces or by using the tab key. Commas are allowed in the universe and sample sizevalues. After the universe file has been selected, the program will return to the input screen.

After opening the data files, the user is returned to the input screen. Click on CONTINUE toresume processing. The user will need to specify the column(s) containing the input data and thetotal number of columns in the data file using the screen shown next. After entering the columninformation, click on OK. The program will resume processing.


(Rev. 11/2005) Page 4-149


The sample data and universe/sample size information must be stored in two tables within thesame Access database. Select the name of the database containing the input tables in thepreceding Open Data File step. This database must have the standard Access extension(.MDB). The name of the database for this illustration is C:\TEMP\VARIABLE.MDB. Theuser may double-click on the VARIABLE database name or single-click on it and the Openbutton on the standard Windows “Open” file screen.

Universe/Sample Sizes Table

The name of the Access table containing the universe/sample size information for thisillustration is UnivPost. The following table shows the table contents:

Sample Data Table

The name of the Access table for this illustration is DATAPOST. The following table shows thefirst five rows. The field name for the first column (“Line-Number” in the illustration) isarbitrary and is not used by the program at any point.


Page 4-150 (Rev. 11/2005)


This data file contains 25 observations in each stratum. The last three rows of the first stratumand the first two rows of the second stratum are shown below:

After opening the database, the user will be asked to select the name of the table containing theuniverse/sample information and the table containing the sample data using the following form. For the universe/sample sizes table, click on the down arrow under Select the table. Afterselecting a table from the drop-down list (UnivPost for this illustration), click on Click here tosee field names. This list will contain the field names for this table. To select the field names,click on the field name for the field containing the universe sizes (“Field2” in this illustration)and click on the box labeled “Universe Sizes.” The field name will appear in this box. Repeatthis procedure for the field containing the sample sizes (“Field3” in this illustration).


(Rev. 11/2005) Page 4-151

To select the sample data field names, first select the table (DATAPOST in this illustration) andclick on the field name for the field containing the first piece of information in the input file(“Difference” in this illustration), then click on the box labeled “Difference Values.” The fieldname will appear in this box. Repeat this procedure for any remaining field names in this table(there are none for this illustration). When all the field names have been specified, click on OK. The program will return to the input screen. NOTE: When the user returns to the input screen,the data file format (Difference Values for this illustration) will be selected based on responseswithin the preceding Access Table and Field Names window. When using an Access input file,the user cannot change this data file format option after returning to the input screen.


With this option, the sample data and universe/sample sizes information must be stored in twoExcel spreadsheets within the same Excel file. Select the name of the file containing bothspreadsheets in the preceding Open Data File step. This file must have the standard Excelextension (.XLS). For this illustration, Excel file DATAPOST.XLS will be used.


Page 4-152 (Rev. 11/2005)

Universe/Sample Size Spreadsheet

The contents of the spreadsheet containing the universe/sample sizes (named “Sizes” in thisillustration) are shown below:


The name of the second spreadsheet containing the sample data is DATAPOST in thisillustration. The first five observations in this spreadsheet are shown below. The data filecontains 25 observation in each stratum for a total of 51 rows (including the first row withlabels). This particular file contains line numbers (1, 2, 3, . . .) in column A. The line numbersare optional.



(Rev. 11/2005) Page 4-153





Page 4-154 (Rev. 11/2005)

Output Options




After selecting the output format, click on CONTINUE. The next screen to appear is the datafile summary, shown next. At this point the user should reconcile the values to determine thatthe data file is complete and accurate. Click on OK to continue or EXIT to exit the program.


(Rev. 11/2005) Page 4-155

Program Output

For the examined, adjusted, and difference sections of the output, the following pieces ofinformation will be displayed for each stratum. The precision information is given at the two-sided 80%, 90%, and 95% confidence levels.

SAMPLE SIZE

The number of sample items belonging to this particular stratum(stratum sample size) or in the combined sample (overall sample size).

UNIVERSE SIZE

The number of universe items for this particular stratum (stratumuniverse size) or in the entire universe (overall universe size).

MEAN The average value for the sample items appraised within a stratum. It isobtained by summing the sample items for this stratum and dividing theresult by the number of sample items within this stratum.

STANDARDDEVIATION

A measurement of the variation of the sample items within a stratumabout the stratum mean.


A measurement of the variation of the estimated stratum universe totalwith respect to all possible estimated totals.

OVERALLSTANDARDERROR(TOTAL)

A measurement of the variation of the point estimate of the overalluniverse total with respect to all possible totals.

POINTESTIMATE

For each stratum, the estimate of the stratum total. It is found bymultiplying the sample mean and the number of universe items for thisstratum. The overall point estimate is the sum of the strata pointestimates.

CONFIDENCELEVEL

The confidence (80%, 90%, 95%) associated with the ability of thecorresponding interval to contain the true mean (or universe total).




Page 4-156 (Rev. 11/2005)

STRATUMPRECISIONAMOUNT

A measurement of the closeness of the sample estimate of the stratumtotal and the actual stratum total. The precision amount is calculated bymultiplying the stratum standard error by the appropriate factor (“Z”value) corresponding to the desired confidence level. For the examined(reviewed) appraisal, the stratum total may be known and should bereviewed by the user to see if, in fact, the actual value does fall withinthe confidence interval.

OVERALLPRECISIONAMOUNT

A measurement of the closeness of the sample estimate of the universetotal and the corresponding unknown universe value. The precisionamount is calculated by multiplying the overall standard error by theappropriate factor (“Z” value) corresponding to the desired confidencelevel. For the examined (reviewed) appraisal, the universe total may beknown and should be reviewed by the user to see if, in fact, the actualvalue does fall within the confidence interval.

PRECISIONPERCENT




Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTPOST.TXT, shown next. The printer output is identical.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 POSTSTRATIFIED VARIABLE APPRAISAL Time: 12:45 AUDIT/REVIEW: Variable - Poststratification

DATA FILE USED: C:\TEMP\DATAPOST.TXT

----------------------D I F F E R E N C E-----------------------Stratum 1 SAMPLE SIZE / UNIVERSE SIZE 25 15,000 MEAN 99.24 STANDARD DEVIATION 26.33 STANDARD ERROR (TOTAL) 61,226.65 POINT ESTIMATE 1,488,600


(Rev. 11/2005) Page 4-157




Stratum 2 SAMPLE SIZE / UNIVERSE SIZE 25 3,000 MEAN 311.40 STANDARD DEVIATION 39.64 STANDARD ERROR (TOTAL) 43,154.68 POINT ESTIMATE 934,200


90% CONFIDENCE LEVEL LOWER LIMIT 863,217 UPPER LIMIT 1,005,183 PRECISION AMOUNT 70,983 PRECISION PERCENT 7.60% Z-VALUE USED 1.644853626951

95% CONFIDENCE LEVEL LOWER LIMIT 849,618 UPPER LIMIT 1,018,782 PRECISION AMOUNT 84,582 PRECISION PERCENT 9.05% Z-VALUE USED 1.959963984540

OVERALL SAMPLE SIZE / UNIVERSE SIZE 50 18,000 POINT ESTIMATE 2,422,800 STANDARD ERROR (TOTAL) 74,907


Page 4-158 (Rev. 11/2005)






Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedis text file or printer. The screen below is the summary of the difference values for the firststratum in this illustration. If the user created a data file containing two values per sample item(e.g., examined and audited values), the summary for each amount (examined, audited,difference) can be obtained by clicking on Additional Summary Info at the bottom of this form. For this illustration, the Additional Summary Info button is not visible since only thedifference values were used in the data file. To obtain the results for the second stratum, click onNext Stratum. The user can click on Next Stratum and Previous Stratum to review theresults for the individual strata. To obtain the overall results, click on OVERALL. Theresulting summary screen immediately follows the summary for the first stratum.


(Rev. 11/2005) Page 4-159


Page 4-160 (Rev. 11/2005)

RAT-STATS User’s Guide VARIABLE - UNKNOWN UNIVERSE SIZE

(Rev. 11/2005) Page 4-161


Purpose

This program calculates the overall precision for a population whose size is unknown. Thisprogram requires that two samples have already been taken and appraised. One sample was usedto estimate the population size and the other sample was taken to estimate one or more variablecharacteristics. The two samples must be appraised prior to executing this module, since thisprogram will ask for the mean and standard deviation of each sample.

Input Screen

VARIABLE - UNKNOWN UNIVERSE SIZE RAT-STATS User’s Guide

Page 4-162 (Rev. 11/2005)




Specify Input Information

The initial screen contains a warning to the user that prior appraisals of the two samples must betaken before running this module. To enter input information, click on Specify InputInformation. The following form will appear:


(Rev. 11/2005) Page 4-163

Sample Used to Estimate the Universe Size

Universe size from which items were sampled

The universe of interest is a subset of some other universe. This larger sampling frame could befile drawers or pages in a check register. What the user has done is sampled from this universe(e.g., file drawers) and counted the number of items that meet the criteria for sample selection inthe other sample. The results for each unit in this sample (e.g., file drawers) is entered into adata file for use in the variable appraisal program. The total number of sample units (e.g., filedrawers) in this universe must be known.

Sample size

The user enters the number of sampling units (e.g., file drawers) drawn from the universe enteredabove.

Sample mean

The variable appraisal program will generate a mean for the sample. The user should enter thatmean value.

Sample standard deviation

The variable appraisal program will generate a standard deviation for the sample. The usershould enter that standard deviation value.

Sample Used for Variable Estimation

Sample size

The user has drawn a second sample of items that meet the criteria for review. The user shouldenter the size of this second sample.

Sample mean

The variable appraisal program previously used with this sample generated a sample mean. Theuser should enter that mean value.


Page 4-164 (Rev. 11/2005)

Sample standard deviation

The variable appraisal program previously used with this sample generated a sample standarddeviation. The user should enter that standard deviation value.

Output Options




Program Output

Based on the data given by the user, the sections of information will be generated by theprogram. The first section displays the information provided by the user.

UNIVERSE The size of the universe from which the sample was drawn to estimatethe population size.

SAMPLE The two sample sizes that were drawn for this evaluation.

MEAN The two mean values entered by the user.

STANDARDDEVIATION

The two standard deviation values entered by the user.


(Rev. 11/2005) Page 4-165

The second section of the output displays the results of the estimation. The precisioninformation is given at the two-sided 80%, 90%, and 95% confidence levels.

POINT ESTIMATE The estimate of the universe total.

STANDARDERROR

A measurement of the variation of the point estimate of the total withrespect to all possible totals for this universe and sample sizes.



PRECISIONAMOUNT

A measurement of the closeness of the sample estimate of the universetotal and the corresponding unknown universe value. For theexamined (reviewed) appraisal, the universe total may be known andshould be reviewed by the user to see if, in fact, the actual value doesfall within the confidence interval.

PRECISIONPERCENT




Page 4-166 (Rev. 11/2005)



Using the specified sample results in the initial screen, the appraisal results were stored inC:\TEMP\OUTUNKNOWN.TXT, shown next. The printer output is identical.

DEPARTMENT OF HEALTH & HUMAN SERVICES OIG - OFFICE OF AUDIT SERVICESDate: 10/22/2004 VARIABLE APPRAISAL WITH UNKNOWN UNIVERSE SIZE Time: 11:48 AUDIT/REVIEW: Variable Unknown Universe Size

= = = = = = = = = = = = = = = = I N P U T = = = = = = = = = = = = = = = = SAMPLE TO SAMPLE FOR ESTIMATE POPULATION VARIABLE ATTRIBUTE UNIVERSE 575 SAMPLE 70 55 MEAN 10.33 15.65 STANDARD DEVIATION 2.75 5.45

= = = = = = = = = = = = = E S T I M A T I O N = = = = = = = = = = = = = 80% CONFIDENCE 90% CONFIDENCE 95% CONFIDENCE POINT ESTIMATE 92,957 92,957 92,957 STANDARD ERROR 5,152 5,152 5,152 LOWER LIMIT 86,355 84,483 82,859 UPPER LIMIT 99,560 101,431 103,055 PRECISION AMOUNT 6,603 8,474 10,098 PRECISION PERCENT 7.10% 9.12% 10.86% Z-VALUE USED 1.281551565545 1.644853626951 1.959963984540

Output to Screen

The program always concludes with a summary on the screen, even if the output option selectedis text file or printer. The screen below is the output screen for this illustration. The usersupplied input is on the left side (labeled INPUT) and the remainder of the output form containsthe appraisal results.


(Rev. 11/2005) Page 4-167


Sample Size Determination

RAT-STATS User’s Guide SAMPLE SIZE DETERMINATION - OVERVIEW

(Rev. 11/2005) Page 5-1

VARIABLE SAMPLE SIZE DETERMINATION - - Unrestricted - Using Reported Amounts - - Unrestricted - Using Estimated Error Rate - - Stratified


OVERVIEW

The purpose of the sample size determination module is to estimate the necessary sample size fora certain precision at a given confidence level. The program will generate optimum sample sizesfor unrestricted and stratified variable samples and attribute samples. A brief example of whento use each module is given below. A detailed explanation of how to use each module isincluded later in this section.

Variable Sample Size Determination

The Variable Sample Size Determination program allows the user to estimate sample sizes forspecified precision percentages and specified confidence levels. In the Variable Unrestricted(Using Reported Amounts) module, the user will have the option of having the program read aprobe sample file to obtain an estimate of the universe mean and standard deviation or inputthese two estimates directly without reading a probe sample file. The Variable Unrestricted(Using Estimated Error Rate) module determines an optimum sample size by first estimating themean and standard deviation of the difference amounts using the reported amounts and anestimated error rate. The Variable Stratified module will determine sample sizes for situationswhere the total sample size is both predetermined or unknown.

Attribute Sample Size Determination

The Attribute Sample Size Determination program determines the sample size when obtaining anattribute simple random sample. The sample size is determined to provide for a specified degreeof precision (using the desired width of the confidence interval) at four levels of confidence(80%, 90%, 95%, and 99%). The resulting sample sizes are the smallest sample sizes capable ofmeeting the specified precision requirement at the stated confidence level.

VARIABLE SAMPLE SIZE DETERMINATION - UNRESTRICTED RAT-STATS User’s Guide

Page 5-2 (Rev. 11/2005)

VARIABLE SAMPLE SIZE DETERMINATIONUNRESTRICTED - Using a Probe Sample

Purpose

This program allows the user to estimate sample sizes for specified precision percentages andspecified confidence levels. The user will have the option of having the program read a probesample file to obtain an estimate of the universe mean and standard deviation or input these twoestimates directly without reading a probe sample file.

Input Screen

The input screen for this program is shown below:

RAT-STATS User’s Guide VARIABLE SAMPLE SIZE DETERMINATION - UNRESTRICTED

(Rev. 11/2005) Page 5-3

Probe Sample

The user has the option of having the program read a probe sample file to obtain an estimate ofthe universe mean and standard deviation or inputting these two estimates directly withoutreading a probe sample file. The probe sample can be contained in a text file, an Excelspreadsheet, or a table within an Access database.

Probe Sample in a Text File

If the probe sample is contained in a text file, click on the OPEN FILE button and select thename of this file. By clicking on the Open button on the file select form, the user will see thefull input screen for this program as shown below:

Full input screenfor Sample SizeDeterminationprogram.


Page 5-4 (Rev. 11/2005)

The file used for this illustration is in C:\TEMP\SAMPDATA.TXT and contains 25observations, shown below. This text file should be in a single column with one sample valueper line. The mean of this sample is 400 and the standard deviation is 50.

321382453459343388313420407 The probe text file in C:\TEMP\SAMPDATA.TXT.395441 (mean = 400, standard deviation = 50)448447333357395477391356368376350461472447

Probe Sample in an Excel Spreadsheet

If the probe sample is contained in an Excel spreadsheet, click on the OPEN FILE button andselect the name of this file. By clicking on the Open button on the file select form, the user willsee the full input screen for this program, shown previously. The Excel file used for thisillustration is C:\TEMP\SAMPDATA.XLS and contains the same 25 observations. The mean ofthis sample is 400 and the standard deviation is 50. This particular file contains labels (variablenames) in the first row and contains line numbers (1, 2, 3, . . .) in column A. The correspondingoptions were selected in the Excel Input screen shown next. The line numbers are optional. Hadcolumn A contained the probe sample data, the second option in the Column A frame in theExcel Input screen should have been selected.


(Rev. 11/2005) Page 5-5


Page 5-6 (Rev. 11/2005)

The probe Excel file inC:\TEMP\SAMPDATA.XLS.

Probe Sample in an Access Database Table

If the probe sample is contained in an Access database table, click on the OPEN FILE buttonand select the name of this database. The Access database used for this illustration isC:\TEMP\SampleSize.MDB and contains a table with 25 observations; the following tableshows the first 18 rows. After clicking on the OPEN FILE button, the user will be asked toselect the name of the table within the selected database using the form shown immediately afterthe table. Click on the down arrow under Select a table. After selecting a table from the drop-down list (SAMPDATA for this illustration), click on Click here to see field names. This listwill contain the field names for this table. To select the field name, click on the field name forthe field containing the probe sample (Data in this illustration), then click on the box alongsideFIELD NAME. Click on OK to continue processing.


(Rev. 11/2005) Page 5-7

The first 18 rows of Access tableSAMPDATA in Access databaseC:\TEMP\SampleSize.MDB.


Page 5-8 (Rev. 11/2005)

No Probe Sample File Used

The user has the option of not using a probe sample in this analysis. Using this option, the userwill be asked to specify the anticipated mean for the sample. If the mean is not known, the bestestimate of the mean may be used. Other sources of data, such as prior reviews, may provideassistance in estimating the mean. Next, the user will be asked to specify the anticipatedstandard deviation for the sample. This may be the hardest value for the user to approximate. As a guide, approximately two-thirds of the sample values lie between the mean plus or minusone standard deviation. For example, if the user specifies a mean of $400 and a standarddeviation of $50, two-thirds of the sample values should lie between $350 and $450. Also,nearly all the sample values should lie within plus or minus three standard deviations of thesample mean (between $250 and $550 for this example).

When the standard deviation is not known, the user has several alternatives for approximating it. Statistical Auditing by Donald Roberts includes several methods for approximating the samplestandard deviation.

When this option is selected the user will see the following message:

By clicking on OK, the user will return to the full input screen for this program, shown in theProbe Sample in a Text File section.

After selecting the desired confidence levels, precision percentages, and output types, the userwill click on OK to continue. The user will be asked to enter the estimated mean in the boxshown below:


(Rev. 11/2005) Page 5-9

Also, the user will be asked to enter the estimated standard deviation in a similar input box.

Working With the Full Input Screen

Confidence Level

The user may select any combination of the following confidence levels: 80%, 90%, 95%, and99%. Selecting all four confidence levels can be done by clicking on the “All” option. If noneof the options are selected, the program will use the default “All” option.

Precision

Sample precision is measured as a percentage. The available precision percentages are 1%, 2%,5%, 10%, 20%, and “Other.” By selecting “All,” the user will obtain the sample sizes for thefirst five precision percentages. When selecting “Other,” the user will be prompted to enter thedesired precision percentage. Enter this value as an integer (e.g., 25 for 25%, 35 for 35%). Afterentering this value, on the screen the user will see the word “Other” change to the specifiedvalue. If none of the options are selected, the program will use the default “All” option.

A confidence interval for the universe mean is obtained by adding and subtracting an amount(say, E) to/from the sample mean. The value of E is determined using the universe size, samplesize, standard deviation, and the selected confidence level. The sample precision percentage iscalculated as E divided by the point estimate of the universe mean times 100.


Page 5-10 (Rev. 11/2005)

Program Output

The output options are a text file and screen; a printer and screen; a text file, printer, and screen;or screen only. The program always finishes with the screen output.


The output for each cell in the output table will consist either the necessary sample size or thetext “- - -.” The necessary sample size is the number of sample items necessary to obtain thespecified sample precision at the specified confidence level. For example, in this illustration, asample size of 106 is necessary to obtain a point estimate having a precision percentage of plusor minus 2% using a 90% confidence level. If the calculated sample size is zero, a text value of“- - -” will appear in this cell. This occurred in the lower left cell for the sample illustration.

The output also contains the estimated mean and standard deviation, along with the specifieduniverse size.


Using the probe sample in C:\TEMP\SAMPDATA.TXT, the sample sizes were saved inC:\TEMP\OUTSIZES.TXT, shown below. The printer output is identical. If any of the samplesizes are under 30, the note shown following the calculated sample sizes is the final portion ofthe program output.


(Rev. 11/2005) Page 5-11

NOTE: Example is for illustrative purposes only. Be sure to use samplesizes that conform to the organization’s minimum sample size standards.


Confidence Level 80% 90% 95% 99% 1% 256 421 597 1026 2% 64 106 150 259Precision 5% 10 (*) 17 (*) 24 (*) 41Level 10% 3 (*) 4 (*) 6 (*) 10 (*) 15% 1 (*) 2 (*) 3 (*) 5 (*) 25% --- 1 (*) 1 (*) 2 (*)

Estimated Mean: 400.00

Estimated Std. Deviation: 50.00


NOTE (*): One or more sample sizes were under 30. The generated samplesizes were the result of mathematical formulas and did notincorporate management decisions concerning the purpose of thesample or current organizational sampling policies. You may needto increase the sample sizes in order to be in compliance withorganizational objectives.

Output to Screen

The program always concludes with a screen summary. The screen output for this illustration isshown below. A universe size of 100,000 is used and the option “All” was selected for theconfidence levels and precision percentages. The “Other” precision percentage was specified as25%. The note on the right side of this screen will appear whenever one or more of the samplesizes are under 30.


Page 5-12 (Rev. 11/2005)



(Rev. 11/2005) Page 5-13

VARIABLE SAMPLE SIZE DETERMINATIONUNRESTRICTED - Using Expected Error Rate

Purpose

This program allows the user to estimate sample sizes for specified precision percentages andspecified confidence levels. The program determines an optimum sample size by first estimatingthe mean and standard deviation of the difference amounts using the reported amounts and anestimated error rate.

Input Screen



Page 5-14 (Rev. 11/2005)

Assumptions

This procedure estimates the mean and standard deviation of the difference (error) amounts byassuming (1) any item found to be in error is 100% in error and (2) the mean and standarddeviation of the nonzero error amounts is the same as the mean and standard deviation of thereported (examined) amounts. Even though these assumptions may not be entirely true, thisprocedure will often give more reliable sample size estimates than those obtained using theVariable Unrestricted (Using Reported Amounts) module since the expected number of zerovalues in the error population is factored into the sample size calculation. The mean andstandard deviation of the error amounts are estimated by assuming the percentage of nonzeroerrors in the error population is equal to the expected error rate (one of the input values) and thenonzero errors resemble the reported amounts; that is, the mean and standard deviation of thenonzero errors are equal to the mean and standard deviation of the reported amounts.

Screen Input

Confidence Level


Precision


A confidence interval for the universe mean is obtained by adding and subtracting an amount(say, E) to/from the sample mean. The value of E is determined using the universe size, samplesize, standard deviation, and the selected confidence level. The sample precision percentage iscalculated as E divided by the point estimate of the universe mean times 100.


(Rev. 11/2005) Page 5-15

Example

The estimated error rate is 15% for a universe of 10,000 transactions. The total reported amountis $3,000,000 and the standard deviation of the reported amounts is $125. Consequently, themean reported amount is $300. Of interest is the required sample size necessary in order toobtain plus or minus 15% using a 90% confidence level. The corresponding input screen followswhere 25% was specified for the “Other” precision level.


Page 5-16 (Rev. 11/2005)

Program Output



The output for each cell in the output table will consist either the necessary sample size or thetext “- - -.” The necessary sample size is the number of sample items necessary to obtain thespecified sample precision at the specified confidence level. In this illustration a sample size of287 is necessary to obtain a point estimate having a precision percentage of plus or minus 15%using a 90% confidence level. If the calculated sample size is zero, a text value of “- - -” willappear in this cell.

The output also contains the estimated mean and standard deviation of the difference (error)values. For this illustration, the estimated mean and standard deviation are $45.00 and $117.55,respectively.


The output results were saved in C:\TEMP\OUTSAMPLE.TXT, shown on the next page. Theprinter output is identical. Notice that obtaining precision levels of 10% or less will requireextremely large sample sizes.


(Rev. 11/2005) Page 5-17



Confidence Level 80% 90% 95% 99% 1% 9181 9486 9633 9784 2% 7370 8219 8676 9188Precision 5% 3095 4248 5119 6443Level 10% 1008 1559 2077 3117 15% 474 758 1044 1675 25% 176 287 403 675


Anticipated Error Rate: 15%

Reported Amounts - - Total Amount: 3,000,000.00

Standard Deviation: 125.00

Difference Values - - Estimated Mean: 45.00

Estimated Standard Deviation: 117.55

If any of the sample sizes are under 30, the note shown below will be the final portion of theprogram output.



Page 5-18 (Rev. 11/2005)

Output to Screen

The program always concludes with a screen summary. The screen output for this illustration isshown below. A note will appear on the right side of this screen whenever one or more of thesample sizes are under 30.

RAT-STATS User’s Guide VARIABLE SAMPLE SIZE DETERMINATION - STRATIFIED

(Rev. 11/2005) Page 5-19

VARIABLE SAMPLE SIZE DETERMINATIONSTRATIFIED

Purpose

This program allows the user to estimate sample sizes for specified precision percentages andspecified confidence levels. This program will generate sample sizes for stratified samples. Thetotal sample size may be determined by the program or specified by the user.

Input Screen


VARIABLE SAMPLE SIZE DETERMINATION - STRATIFIED RAT-STATS User’s Guide

Page 5-20 (Rev. 11/2005)

Sample Size is Known/Sample Size is Unknown

If the user plans on performing a stratified sample and the overall sample size has beenpredetermined, click on “Sample size is known: Determine the optimal allocation.” Enter thetotal sample size in the box shown below:

If the overall sample size has not been predetermined, click on “Total sample size is unknown.”

Working With the Full Input Screen

Confidence Level


Precision


Program Output



(Rev. 11/2005) Page 5-21

After clicking on OK, the user will need to specify information for each of the strata using thefollowing form. To move to the next stratum, click on NEXT. The input values for eachstratum may be viewed by clicking on NEXT and PREVIOUS to move from one stratum toanother. When all input values have been entered, click on OK.


The output for each cell in the output table will consist of either the necessary sample size or thetext “- - -.” The necessary sample size is the number of sample items necessary to obtain thespecified sample precision at the specified confidence level. For example, in this illustration, inthe first stratum, a sample size of 111 is necessary to obtain a point estimate having a precisionpercentage of plus or minus 5% using a 90% confidence level. If the calculated sample size iszero, a text value of “- - -” will appear in this cell.

The output also contains the user-specified estimated mean, estimated standard deviation, andestimated universe size. The computed ratio of the total sample size allocated to this stratum isalso contained in the output.

Program Output—Total Sample Size is Unknown

Output to a Text File or PrinterThe sample sizes for this illustration were saved in C:\TEMP\OUTSTRSIZES.TXT, shownbelow. The printer output is identical. The option “All” was selected for the confidence levelsand precision percentages and the “Other” precision percentage was specified as 25%.


Page 5-22 (Rev. 11/2005)

NOTES: (1) The program calculates the estimated mean and standard deviation of the entireuniverse ($5,8333.33 and $4,579.54, respectively, in this illustration).

(2) Whenever one or more of the sample sizes are under 30, the program output willconclude with the note immediately following the calculated total sample sizes.



NBR DESCRIPTION -- MEAN -- -- STD.DEV. -- -- UNIVERSE -- -- RATIO -- 1 High Income 10,000.00 5,000.00 100,000 20.00% 2 Low Income 5,000.00 4,000.00 500,000 80.00%

- TOTALS - 5,833.33 4,579.54 600,000= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

Sample Sizes for Stratum 1: High Income

Confidence Level 80% 90% 95% 99% 1% 1653 2699 3795 6406 2% 418 687 972 1669Precision 5% 67 111 157 271Level 10% 17 (*) 28 (*) 40 68 15% 8 (*) 13 (*) 18 (*) 31 25% 3 (*) 5 (*) 7 (*) 11 (*)

Sample Sizes for Stratum 2: Low Income

Confidence Level 80% 90% 95% 99% 1% 6611 10793 15180 25624 2% 1671 2745 3888 6676Precision 5% 268 442 627 1081Level 10% 68 111 157 271 15% 30 50 70 121 25% 11 (*) 18 (*) 26 (*) 44

Total Sample Sizes

Confidence Level 80% 90% 95% 99% 1% 8264 13492 18975 32030 2% 2089 3432 4860 8345Precision 5% 335 553 784 1352Level 10% 85 139 197 339 15% 38 63 88 152 25% 14 (*) 23 (*) 33 55


(Rev. 11/2005) Page 5-23



If any of the calculated samples sizes exceeds the corresponding universe size, the program willconclude with the following reminder:



Page 5-24 (Rev. 11/2005)


Output to Screen

The program always concludes with a screen summary. The screen output for the first stratum inthis illustration is shown next. The option “All” was selected for the confidence levels andprecision percentages. The “Other” precision percentage was specified as 25%. To view theoutput for the second stratum, click on NEXT and to see the overall results (total sample sizes),click on Total Sample. The window that appears when clicking on “Click here for commentsregarding these sample sizes” immediately follows the two output screens. If any of thecalculated samples sizes exceeds the corresponding universe size, the comments window willalso contain NOTE (#) from the Output to a Text File or Printer section.


(Rev. 11/2005) Page 5-25

NOTE: The program calculates the estimated mean and standard deviation of the entire universe($5,8333.33 and $4,579.54, respectively, in this illustration).


Page 5-26 (Rev. 11/2005)


Program Output—Total Sample Size is Known


The sample sizes for this illustration were saved in C:\TEMP\OUTSTRKNOWN.TXT, shownnext. The printer output is identical. The total sample size was specified as 500. The option“All” was selected for the confidence levels.

NOTE: The program calculates the estimated mean and standard deviation of the entire universe($5,8333.33 and $4,579.54, respectively, in this illustration). The calculated sample sizes are100 (High Income stratum) and 400 (Low Income stratum).



NBR DESCRIPTION -- MEAN -- -- STD.DEV. -- -- UNIVERSE -- 1 High Income 10,000.00 5,000.00 100,000 2 Low Income 5,000.00 4,000.00 500,000

- TOTALS - 5,833.33 4,579.54 600,000= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =

Precision Values:

Confidence Level 80% 90% 95% 99%Sample Precision 4.09% 5.25% 6.26% 8.22%

The following sample sizes are based on a total sample size of 500.

Stratum 1: High Income Sample Size Ratio 100 20.00%

Stratum 2: Low Income Sample Size Ratio 400 80.00%


(Rev. 11/2005) Page 5-27

If any of the sample sizes are under 30 the text file/printer output will contain the followingreminder:


If any of the calculated samples sizes exceeds the corresponding universe size, the programoutput will contain the following reminder:


If, due to rounding, the total sample size calculated does not equal the total sample sizerequested, the program will contain a reminder similar to the following:

NOTE (!): Due to rounding, the total sample size calculated(499) does not equal the sample size requested (500).

Output to Screen

The program always concludes with a screen summary. The screen output for the first stratum inthis illustration is shown next. The total sample size was specified as 500. The option “All” wasselected for the confidence levels. To view the output for the second stratum, click on NEXTand to see the overall results (total sample sizes), click on Total Sample.

NOTE: The program calculates the estimated mean and standard deviation of the entireuniverse ($5,8333.33 and $4,579.54, respectively, in this illustration). The calculated samplesizes are 100 (High Income stratum) and 400 (Low Income stratum).

If any of the conditions described in NOTES (*), (#), or (!) above exist, the screen output willcontain the following button. By clicking on this button, the user will see the correspondingnotes, depending on the existing conditions.


Page 5-28 (Rev. 11/2005)


RAT-STATS User’s Guide ATTRIBUTE SAMPLE SIZE DETERMINATION

(Rev. 11/2005) Page 5-29


Purpose

This program determines the sample size for an attribute simple random sample. The samplesize is determined to provide for a specified degree of precision (using the desired width of theconfidence interval) at four levels of confidence (80%, 90%, 95%, and 99%). The resultingsample sizes are the smallest sample sizes capable of meeting the specified precision requirementat the stated confidence level.

Confidence intervals for attribute sampling are exact and are based on the hypergeometricdistribution. As a result, such confidence intervals are usually not symmetric about the pointestimate. For example, the point estimate may be 3% and the corresponding 90% confidenceinterval is from 2% to 6%. For this illustration, the width of the confidence interval is 4% andthe confidence level is 90%. Consequently, attribute confidence intervals differ from the usualinterval obtained by deriving the point estimate plus or minus the estimated precision, where theestimated precision is half the width of the resulting confidence interval. Because of this, the“desired precision” for the attribute sampling procedure must be specified as the desired width(rather than the half width) of the confidence interval.

The input requirements also include the size of the universe and the anticipated rate ofoccurrence in the universe. This rate of occurrence is generally estimated from past experience,either from similar systems or a past review of this universe. If no information concerning therate of occurrence is available, the most conservative procedure is to specify 50% for this value. If the actual rate of occurrence differs from the user-specified rate of occurrence, this in no wayaffects the sample’s validity; however, the resulting precision (confidence interval width) willlikely differ from the specified “desired precision.”

Input Screen

The input screen for this program is shown next.

ATTRIBUTE SAMPLE SIZE DETERMINATION RAT-STATS User’s Guide

Page 5-30 (Rev. 11/2005)

Input Values

Confidence Level

The user may select any combination of the following confidence levels: 80%, 90%, 95%, and99%. Selecting all four confidence levels can be done by clicking on the “All” option.

Anticipated Rate of Occurrence

This value is the expected rate of occurrence for the universe. It should be expressed as apercentage (e.g., enter “15” for 15%, not “.15”). If no information is available for the anticipatedrate of occurrence, the most conservative procedure is to specify a rate of occurrence of 50%.


(Rev. 11/2005) Page 5-31

Universe Size

The universe size is the total number of items from which the sampled items were selected. Thisnumber should be entered without commas (e.g., 50000, not 50,000). The program will insertcommas upon leaving this box.

Desired Precision Range

This value is the desired width of the resulting confidence interval, expressed as a percentage. Itis equal to the upper confidence limit minus the lower limit. For example, if the confidenceinterval 10% to 15% satisfies the user’s precision requirements, the response to this query wouldbe “5.”

Program Output



The output for each cell in the output table will consist of either the necessary sample size or thetext “- - - .” The necessary sample size is the number of sample items necessary to obtain thespecified sample precision at each confidence level. For example, in this illustration, a samplesize of 680 is necessary to obtain a confidence interval having a width of 5% using a 90%confidence level. If the calculated sample size is zero, a text value of “- - -” will appear in thiscell.

The output also contains the user-specified anticipated rate of occurrence, desired precisionrange, and universe size.

ATTRIBUTE SAMPLE SIZE DETERMINATION RAT-STATS User’s Guide

Page 5-32 (Rev. 11/2005)



The sample sizes for this illustration were saved in C:\TEMP\OUTSIZES.TXT, shown below. The printer output is identical. The input values are those shown on the initial input screenwhere the option “All” was selected for the confidence levels.


Confidence Level 80% 90% 95% 99%Sample Size 439 680 926 1,480




If any of the samples sizes are under 30, the following note will be the final part of the text fileand/or printer output:



(Rev. 11/2005) Page 5-33


Output to Screen

The program always concludes with a screen summary. The screen output for this illustration isshown below. If one or more sample sizes are under 30, these sample sizes are flagged using“(*)” (e.g., 22 (*)) and the following note will appear in the output screen:

Appendices

RAT-STATS User’s Guide APPENDIX A - DATA LIMITATIONS

(Rev. 11/2005) Page A-1

Data Limitations for RAT-STATS 2007

Random Number Modules

Module Limitation

Single Stage Number of random values (including spares) #10,000

Sets of Two Number of random values (including spares) #10,000

Sets of Three Number of random values (including spares) #10,000

Sets of Four Number of random values (including spares) #10,000

Frames - Single Stage Number of frames in the universe # 50Number of random values (including spares) #10,000

Frames - Sets of Two Number of frames in the universe # 50Number of random values (including spares) #10,000

RHC Sample Selection Number of rows in input file # 6,000Number of units in the sample # 3,000

Attribute Modules

Module Limitation

Unrestricted Universe size # 2,147,483,647

Stratified Number of strata # 125

2-Stage Unrestricted Number of primary units in the sample # 250

3-Stage Unrestricted Number of primary units in the sample # 50Total number of secondary units in the sample # 50,000

APPENDIX A - DATA LIMITATIONS RAT-STATS User’s Guide

Page A-2 (Rev. 11/2005)

RHC 2 Stage Number of primary units in the sample # 3,000

RHC 3 Stage Number of primary units in the sample # 50Total number of secondary units in the sample # 4,000

Stratified Cluster Number of strata # 100Number of clusters within each stratum # 500

Stratified Multistage Number of strata # 500Universe size for each stratum < 1,000,000,000

Variable Modules

Module Limitation

Unrestricted Universe size < 1,000,000,000

Stratified Number of strata #50

2-Stage Unrestricted Number of primary units in the sample # 500Number of primary units in the universe < 1,000,000Number of secondary units in the universe < 1,000,000,000

3-Stage Unrestricted Number of primary units in the sample # 50Total number of primary units in the universe # 30,000Total number of secondary units in the sample # 50,000

RHC 2 Stage Number of primary units in the sample # 3,000

RHC 3 Stage Number of primary units in the sample # 50Total number of secondary units in the sample # 4,000

Stratified Cluster Number of strata # 100Number of clusters within each stratum # 500

Stratified Multistage Number of strata # 500

Poststratified Number of strata # 150

RAT-STATS User’s Guide APPENDIX A - DATA LIMITATIONS

(Rev. 11/2005) Page A-3

Unknown Univ. Size Universe size sample: Universe size < 1,000,000,000 Sample size < 1,000,000

Variable estimation sample: Sample size < 1,000,000

Sample Size Modules

Module Limitation

Variable Unrestricted (Using a Probe Sample)

Probe sample size must be #500,000 (all formats)

Variable Stratified Number of strata #500

Attribute Unrestricted Anticipated rate of occurrence $ .5%Anticipated rate of occurrence # 98%Desired precision range $ 1%Desired precision range # 99%

RAT-STATS User’s Guide APPENDIX B - DATA LIMITATIONS

(Rev. 11/2005) Page B-1

Numerical Accuracy in RAT-STATS 2007

Attribute Modules

Module Limitation

Unrestricted Projected items in the universe and the universe totalconfidence limits are accurate to the nearest integer, providedthe number of digits does not exceed 12. Universe proportionpoint estimates and the associated confidence limits areaccurate to the nearest .05 percent. Calculation of thehypergeometric distribution function is accurate to at least 12significant figures, provided the universe size is less than2,147,483,647.

Stratified Calculations done in this program module are accurate to atleast 12 significant digits. Projected items in the universe andthe universe total confidence limits are accurate to the nearestinteger, provided the number of digits does not exceed 12. Universe proportion point estimates and the associatedconfidence limits are accurate to the nearest .05 percent.

2-Stage Unrestricted Calculations done in this program module are accurate to atleast 12 significant digits. Projected items in the universe andthe universe total confidence limits are accurate to the nearestinteger, provided the number of digits does not exceed 12. Universe proportion point estimates and the associatedconfidence limits are accurate to the nearest .5 percent.

3-Stage Unrestricted Calculations done in this program module are accurate to atleast 12 significant digits. Projected items in the universe andthe universe total confidence limits are accurate to the nearestinteger, provided the number of digits does not exceed 12. Universe proportion point estimates and the associatedconfidence limits are accurate to the nearest .5 percent.

RHC 2 Stage Calculations done in this program module are accurate to atleast 12 significant digits. Projected items in the universe andthe universe total confidence limits are accurate to the nearestinteger, provided the number of digits does not exceed 12.

APPENDIX B - DATA LIMITATIONS RAT-STATS User’s Guide

Page B-2 (Rev. 11/2005)

RHC 3 Stage Calculations done in this program module are accurate to atleast 12 significant digits. Projected items in the universe andthe universe total confidence limits are accurate to the nearestinteger, provided the number of digits does not exceed 12.

Stratified Cluster Calculations done in this program module are accurate to atleast 12 significant digits. Projected items in the universe andthe universe total confidence limits are accurate to the nearestinteger, provided the number of digits does not exceed 12. Universe proportion point estimates and the associatedconfidence limits are accurate to the nearest .5 percent.

Stratified Multistage Calculations done in this program module are accurate to atleast 12 significant digits. Projected items in the universe andthe universe total confidence limits are accurate to the nearestinteger, provided the number of digits does not exceed 12. Universe proportion point estimates and the associatedconfidence limits are accurate to the nearest .5 percent.


(Rev. 11/2005) Page B-3

Variable Modules

Module Limitation

Unrestricted Calculations done in this program module are accurate to atleast 12 significant digits. The descriptive statistics calculatedare accurate to two decimal places, provided the number ofsignificant digits does not exceed 12. The projected items inthe universe and the universe total confidence limits areaccurate to the nearest integer, provided the number of digitsdoes not exceed 12. Calculation of the critical values of the t-distribution are accurate to at least 12 decimal places fordegrees of freedom between 1 and 10,000 and confidence levelsof 80%, 90%, and 95%.

Stratified Calculations done in this program module are accurate to atleast 12 significant digits. The descriptive statistics calculatedare accurate to two decimal places, provided the number ofsignificant digits does not exceed 12. The projected items andthe confidence limits for the universe and for each stratum areaccurate to the nearest integer, provided the number ofsignificant digits does not exceed 12. Calculation of the criticalvalues of the t-distribution are accurate to at least 12 decimalplaces for degrees of freedom between 1 and 10,000 andconfidence levels of 80%, 90%, and 95%.

2-Stage Unrestricted Calculations done in this program module are accurate to atleast 12 significant digits. The projected items in the universeand the universe total confidence limits are accurate to thenearest integer, provided the number of digits does not exceed12. Calculation of the critical values of the standard normaldistribution are accurate to at least 12 decimal places forconfidence levels of 80%, 90%, and 95%.

3-Stage Unrestricted Calculations done in this program module are accurate to atleast 12 significant digits. The projected items in the universeand the universe total confidence limits are accurate to thenearest integer, provided the number of digits does not exceed12. Calculation of the critical values of the standard normaldistribution are accurate to at least 12 decimal places forconfidence levels of 80%, 90%, and 95%.

APPENDIX B - DATA LIMITATIONS RAT-STATS User’s Guide

Page B-4 (Rev. 11/2005)

RHC 2 Stage Calculations done in this program module are accurate to atleast 12 significant digits. The projected items in the universeand the universe total confidence limits are accurate to thenearest integer, provided the number of digits does not exceed12. Calculation of the critical values of the standard normaldistribution are accurate to at least 12 decimal places forconfidence levels of 80%, 90%, and 95%.

RHC 3 Stage Calculations done in this program module are accurate to atleast 12 significant digits. The projected items in the universeand the universe total confidence limits are accurate to thenearest integer, provided the number of digits does not exceed12. Calculation of the critical values of the standard normaldistribution are accurate to at least 12 decimal places forconfidence levels of 80%, 90%, and 95%.

Stratified Cluster Calculations done in this program module are accurate to atleast 12 significant digits. The projected items in the universeand the universe total confidence limits are accurate to thenearest integer, provided the number of digits does not exceed12. Calculation of the critical values of the standard normaldistribution are accurate to at least 12 decimal places forconfidence levels of 80%, 90%, and 95%.

Stratified Multistage Calculations done in this program module are accurate to atleast 12 significant digits. The projected items in the universeand the universe total confidence limits are accurate to thenearest integer, provided the number of digits does not exceed12. Calculation of the critical values of the standard normaldistribution are accurate to at least 12 decimal places forconfidence levels of 80%, 90%, and 95%. Note: The point estimate and confidence limits are accurate tothe nearest integer, subject to the limitation that all input valueshave been rounded to the nearest integer.

Poststratified Calculations done in this program module are accurate to atleast 12 significant digits. The projected items in the universeand the universe total confidence limits are accurate to thenearest integer, provided the number of digits does not exceed12. Calculation of the critical values of the standard normaldistribution are accurate to at least 12 decimal places forconfidence levels of 80%, 90%, and 95%.


(Rev. 11/2005) Page B-5

Unknown Univ. Size Calculations done in this program module are accurate to atleast 12 significant digits. The projected items in the universeand the universe total confidence limits are accurate to thenearest integer, provided the number of digits does not exceed12. Calculation of the critical values of the standard normaldistribution are accurate to at least 12 decimal places forconfidence levels of 80%, 90%, and 95%.

RAT-STATS User’s Guide APPENDIX C - TROUBLESHOOTING

(Rev. 11/2005) Page C-1

Troubleshooting for RAT-STATS 2007

1. When running RAT-STATS 2007, an error “Error 462 - Printer Error” occurs.

Solution. Get into the Control Panel and Click on Printers and Faxes. Set one of youravailable printers as the default printer.

2. When selecting random numbers with output to Excel, the Excel file cannot be viewed.

Solution. The user must first exit RAT-STATS 2007 before opening the Excel spreadsheet.

3. When clicking on the HELP button, an error message occurs.

Solution. The User Guide file is already open. Either click on this file in the taskbar to useit or close the User Guide file before clicking on HELP.

4. When opening an Access database file, an error message occurs.

Solution. The Access file is likely specified as “read only.” Locate this file on yourcomputer, right click on it, left click on Properties, and make sure the Read-onlybox is unchecked.

Date post:	13-Aug-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Medicare and Medicaid Audit Sampling...

Documents