Alpha

Figure 01 has three curves in it, showing three different values for alpha. The three probability curves are all identical, but the value of the sample correlation† (which is the line on the right side of the graph that bisects the population probability curve) increases between them. That increase shifts the curve upward (in our sideways view) and decreases the area under† (where “under” means between the curve and the vertical baseline) the probability curve that is bounded by the null line. That decreasing area is equal to alpha.

Figure 01

This is what you would expect to happen. Remember that alpha† (a.k.a., the “confidence level”, or “p value”) indicates the probability that the population correlation is not null. Which is another way of saying that it measures the chances that the correlation is “real”, and not a result of random error. More importantly alpha is a measure of the chance of making a “Type I” error.

So, the larger the sample correlation is, the less likely it is that no such correlation exists in the population, and that all you have found is random error. If the sample correlation is quite small, it's easy to believe that the correlation is the result of sampling errors, because it wouldn't take much error to create the illusion of a small correlation. But it would take a very large and therefore unlikely amount of sampling error to create the illusion of a large correlation.

Beta

Figure 02 also has three graphs in it, with the statistical power increasing between them, while its inverse, beta, is decreasing.

Figure 02

Alpha is the same for all three graphs, and therefore the minimum sample correlation that is statistically significant is also the same. As the population correlation† (which is the line on the left side of the graph that bisects the sample probability curve) gets higher in the sideways graph, it gets farther away from the minimum statistically significant sample correlation, and the proportion of the area under the sample probability curve that's above the minimum correlation gets larger. That area is the statistical power.

The inverse of statistical power is beta; meaning it's the area that's below the minimum correlation. So, as the population correlation increases, beta gets smaller. And beta is the chance of making a Type II error.

Trading alpha for beta

Once you've estimated the effect size† (which in a sideways graph is the same as the “population correlation”, as is discussed later) for a study, and set the number of samples you'll collect, there's only two of the four characteristics of linear regressions left that can be changed: alpha and beta. Figure 03 shows two graphs for a study with a fixed population correlation and sample size. What changes between them is the placement of the population probability curve.

Figure 03

We can move the population probability curve by changing the alpha for the analysis. Despite the nearly universal use of ‘p<.05’, there is nothing mathematically special about that value; we can use a value for alpha other than 0.05 and still get a statistically sound result. What does inevitably change when we change alpha is beta, and the statistical power.

In the sideways graph (Figure 03), if we want to increase alpha, we shift the population probability curve down. That also shift the minimum sample correlation that's statistically significant down. Therefore, less of the area under the sample probability curve is below that minimum correlation line, and more of the area is above it. Which is the same as saying that beta has gone down, and the statistical power has gone up.

So, once effect size and sample size are fixed, alpha and beta are inversely correlated. In other words, the chances of a Type I error is inversely correlated to the chances of a Type II error. So, if you don't care at all about Type II errors, you can minimize the value of alpha by maximizing the value of beta. Which, in practical application, is typically 0.50, which makes the statistical power 50%.