Power and Sample Size Programs
This summary was originally prepared by Steve Shiboski in December, 1999. Last updated in October 2006.
You can view this summary organized by:
The same information is provided in each of the above links, it is just organized differently.
Type of Test
- z-test
- t-test
- ANOVA
- Simple Regression/Correlation
- More General Linear Models
- Logistic Regression
- Proportions
- Chi-Square Contingency Tables
- Survival Analysis
- Non Parametric Analyses
z-test
One or 2 sample comparison of means assuming a normal distribution are done by lots of programs and web sites.
Software:
Web Sites:
- David Schoenfeld's Statistical Considerations for Clinical Trials and Scientific Experiments
- UCLA Calculator Service
- Sample Size and Power -- Survival Outcomes Johns Hopkins SPH
- Rollin Brant's Sample Size Calculators University of Calgary
- Simple Interactive Statistical Analysis
- Russ Lenth's Power and Sample Size University of Iowa
t-test
One or 2 sample comparison of means assuming a t-distribution are done by the following programs and web sites
Software:
Web Sites:
- Russ Lenth's Power and Sample Size University of Iowa
ANOVA
Several programs and web sites perform sample size and power calculations for 1-way and factorial ANOVAs. They differ with respect to how "effect size" is specified - with some it can be generated for you from treatment means that you specify, with others it is calculated as the ratio of between group to within group MSE, and others it is specified via the non-centrality parameter of the F distribution. Usually the null hypothesis is that the treatment means are all equal, but this isn't always the case. Usually treatment effects are assumed to be fixed, but a couple of programs have a "random" choice.
Software:
- G*Power (1-way only)
- NQuery Advisor (1 and 2 way only)
- PASS (1, 2, and 3-way)
- PC-Size (1 and 2 way)
- Splus (very general function to deal with various linear models)
Websites:
- Sample size computation for multiple comparisons Ohio State
- Russ Lenth's Power and Sample Size University of Iowa (1-, 2-, & 3-way, fixed & random)
Simple Regression/Correlation
Effect size/alternative hypothesis is specified in various ways here, sometimes only by the r2 of the model.
Software:
- G*Power
- NQuery Advisor
- PASS
- PC-Size
- PS
- DSTPLAN (correlation)
- Splus (very general function to deal with various linear models)
Websites:
- David Schoenfeld's Statistical Considerations for Clinical Trials and Scientific Experiments
- UCLA Calculator Service
More General Linear Models
A couple of people have contributed functions to StatLib for Splus and SAS.
Logistic Regression
From what I can gather only NQuery Advisor and PASS have this capability.
Proportions
Several programs and websites will take care of this.
Software:
Websites:
- Proportion Difference Power / Sample Size Calculation
- Rollin Brant's Sample Size Calculators University of Calgary
- UCLA Calculator Service
Chi-Square Contingency Tables
Some of these are restricted to the 2x2 or 2xM situation.
Software:
- G*Power (M level goodness of fit only)
- NQuery Advisor (2x2, Fisher's exact test, 2xM)
- PASS (Fisher's exact test, 2xM)
- DSTPLAN (Fisher's exact test, 2xM)
- PS (Fisher's exact test, Chi-square, McNemar)
Websites:
- UCLA Calculator Service (Fisher's exact test and Case/Control)
- David Schoenfeld's Statistical Considerations for Clinical Trials and Scientific Experiments (Fisher's exact test)
(Interestingly I didn't get these to agree.)
Survival Analysis
The input, and the flexibility they exhibit, is really varied. Most assume an exponential survival time.
Software:
Websites:
- Sample Size and Power -- Survival Outcomes Johns Hopkins SPH (for proportional & non-proportional hazards)
Non Parametric Analyses
Only 1 software package dealt with this, PASS (for Wilcoxon & Mann-Whitney tests)
Operating System
The following programs are for use in MS-DOS or Windows environments:
The following work with a Unix operating system:
- Splus
- for various Splus functions for power calculations see:
- http://lib.stat.cmu.edu/S/power
- http://lib.stat.cmu.edu/S/asypow
- David Heitjan's Optimization Functions Univeristy of Pennsylvania
- Stata
- SAS
- for various SAS macros for power calculations see:
- DSTPLAN
Don't forget all the web sites that are OS-independent:
- Sample Size and Power -- Survival Outcomes Johns Hopkins SPH
- UCLA Calculator Service
- David Schoenfeld's Statistical Considerations for Clinical Trials and Scientific Experiments
- Proportion Difference Power / Sample Size Calculation
- Rollin Brant's Sample Size Calculators University of Calgary
- Russ Lenth's Power and Sample Size University of Iowa
- Sample size computation for multiple comparisons Ohio State
- Simple Interactive Statistical Analysis
Cost
The following programs are free:
The following programs cost $:
- NQuery Advisor ($750 with Academic Discount)
- PASS $750
Admittably, these programs do a lot more than the others, without additional programming.
Software/Website
Click on the name of a program to find out more information about it.
Listing of Websites
- G*Power is a free
application that will run under Windows XP/Vista or Mac OS X 10.4. It uses both exact and approximate
methods to calculate power. It will deal with samplesize/power calculations
for t-tests, 1-way ANOVAs, regression, correlation, and chi-square goodness
of fit. The program will calculate effect size for you after inputting
some key information. For t-tests and ANOVAs you can find the effect size
by supplying mean and variance information, for regression and correlation
the effect size is a function of the r2. For all of these the non-centrality
parameter of the appropriate distribution under the alternative is presented
so that one can see how this effect size translates to these terms. There
is some relatively extensive documentation on the web. For more information
see:
http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/ - PC-Size is a free
DOS application that will run under Windows. It uses exact and approximate
methods to calculate power. It will deal with samplesize/power calculations
for t-tests, 1-way and 2-way ANOVA, simple regression, correlation, and
comparison of proportions. The program will calculate effect size under
the alternative with some input. There are three ways to calculate the
alternative in this program: (1) by specifying individuals effects, (2)
specifying a range and (3) specifying a ratio of variances. The non-centrality
parameter of the appropriate distribution under the alternative is presented
so that one can see how the effect size translates to these terms. Documentation
comes with the executable. For more information see: http://www.esf.edu/efb/gibbs/monitor/usingDSTPLANandPCSIZE.pdf
ftp://ftp.simtel.net/pub/simtelnet/msdos/statstcs/size102.zip - DSTPLAN is a free application that will run under Windows, and since fortran source code is provided I guess it can run under Unix as well. It uses approximate methods to calculate power. It will calculate samplesize/power for t-tests, correlation, a difference in proportions, 2xN contingency tables, and various survival analysis designs. There is pretty extensive documentation. For more information see: http://biostatistics.mdanderson.org/SoftwareDownload/SingleSoftware.aspx?Software_Id=41
- Splus will run under both Windows and Unix. It provides functions for samplesize/power for t-tests and comparing proportions. People have contributed functions to StatLib for more complex designs (see websites). The functions can be pretty general, but documentation is spartan. See: http://www.insightful.com/products/splus/default.asp
- NQuery Advisor is a ~$750 program that runs in Windows. It will perform samplesize/ power calculations for t-tests, 1 and 2 way ANOVAS, tests of contrasts in 1-way ANOVAs, univariate repeated measures designs, regression (simple, multiple and logistic), correlation, difference of proportions, 2XN contingency tables, and survival analyses. For more information see: http://www.statsol.ie/nquery/nquery.htm
- PASS is a ~$750 Windows program. It will perform samplesize/power calculations for z-tests, t-tests, 1, 2, and 3-way ANOVAs, univariate repeated measures designs, regression (simple, multiple and logistic), correlations, difference in proportions, 2xN contingency tables, survival analyses and simple non-parametric analyses. For more information see: http://www.ncss.com/pass.html
- PS is a free Windows program that will perform samplesize/power calculations for t-tests, Chi-square, Fisher's exact, McNemar's, simple regression, and survival analysis. To download the program see: http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/PowerSampleSize
- Stata isn't free but we're already using it. It has some simple built-in power and sample size functions. See: http://www.stata.com/