For a one-way blocked ANOVA, the analysis is not seriously affected if there is serial correlation for measurements within a block (across treatments), as long as there is no interest in testing differences between the blocks. In fact, if the blocks are random, then there generally is a correlation for measurements within a block, although it is assumed to be constant.
However, if there is serial correlation for measurements within a treatment (across blocks), then the reported F test is misleading. If the serial correlation is positive, then the actual P value is much greater than the reported P value, making it possible to falsely conclude that there are significant differences when none exist. Conversely, if the serial correlation is negative, then the actual P value is much smaller than the reported P value, making it possible to falsely conclude that there are no significant differences when the treatment means are in fact significantly different.
If the row order of the data reflect the order in which the data were collected, an index plot of the data [data value plotted against row number] can reveal patterns in the plot that could suggest possible time effects. Note that because data for a one-way blocked ANOVA is often arranged systematically with respect to the blocking factor as well as the treatment factor, it may be difficult to detect an implicit factor in the original data this way. Examining the residuals, from which the linear effects of both the treatment factor and the blocking factor have been removed, may be an easier way to uncover an implicit factor.
The F statistic is based on the sample means and the sample variances, each of which is sensitive to outliers. (In other words, neither the sample mean nor the sample variance is resistant to outliers, and thus, neither is the F statistic.) In particular, a large outlier can inflate the overall variance, decreasing the F statistic and thus perhaps eliminating a significant difference. A nonparametric test may be a more powerful test in such a situation. If you find outliers in your data that are not due to correctable errors, you may wish to consult a statistician as to how to proceed.
For data sampled from a normal distribution, normal probability plots should approximate straight lines, and boxplots should be symmetric (median and mean together, in the middle of the box) with no outliers.
Because the sample sizes for a one-way blocked ANOVA are forced to be balanced, the ANOVA's F test will not be much affected even if the population distributions are skewed, For the same reason, the F test will not be seriously affected by light-tailedness or heavy-tailedness, unless the sample sizes are small (less than 5), or the departure from normality is extreme (kurtosis less than -1 or greater than 2), especially if the samples are extremely nonnormal in different ways (e.g., one very skewed to the right while another is very skewed to the left).
Robust statistical tests operate well across a wide variety of distributions. A test can be robust for validity, meaning that it provides P values close to the true ones in the presence of (slight) departures from its assumptions. It may also be robust for efficiency, meaning that it maintains its statistical power (the probability that a true violation of the null hypothesis will be detected by the test) in the presence of those departures. The one-way blocked ANOVA's F test is robust for validity against nonnormality, but it may not be the most powerful test available for a given nonnormal distribution, although it is the most powerful test available when its test assumptions are met. In the case of nonnormality, a nonparametric test or employing a transformation may result in a more powerful test.
If the treatment variances are unequal, then the chance increases slightly of incorrectly reporting a significant difference in the means when none exists. This chance of incorrectly rejecting the null hypothesis is greater when the population variances are very different from each other, particularly if there is one sample variance very much larger than the others.
Conversely, if the block variances are unequal, the chance increases slightly of failing to find a significant difference between the treatment means when they are in fact different, meaning that the ANOVA F test becomes less powerful.
If both nonnormality and unequal variances are present, employing a transformation may be appropriate. A nonparametric test like the Friedman test still assumes that the population variances are comparable.
The inequality of block variances can also be judged by examining the relative size of the variances of the data, categorized by blocks instead of by populations (treatments).
The plot of residuals against fitted values may help detect such interaction. The plot of observed values against sample (treatment) number may be even more useful in detecting interaction. If there is no interaction, the line segments (one for each pair) should be parallel or nearly so.
If there is interaction between blocks and treatments, the F statistic for the test of the treatment factor will tend to decrease, thus making it less likely that a significant difference will be detected. Thus, if the F test produces a significant result when there is interaction, the F test would have been significant without interaction. This means that if the test for treatment differences is significant, you need not worry too much about the effect of interactions on the test results. However, if the test is not significant, there is the chance that the lack of significance was due to interaction between blocks and treatments, despite a genuine difference between the treatments.
A fan pattern like the profile of a megaphone, with a noticeable flare either to the right or to the left as shown in the picture (one or more of the "stacks" of data points is much longer than the others), suggests that the variance in the values increases in the direction the fan pattern widens (usually as the sample mean increases), and this in turn suggests that a transformation may be needed.
Side-by-side boxplots of the samples can
also reveal lack of homogeneity of variances
if some boxplots are much longer than others, and reveal suspected
outliers.
The plot of each sample's values against its block number will also consist of vertical "stacks" of data points, one stack for each block. If the assumptions for the samples' population distributions are correct, the stacks should also be about the same length. Again, outliers may appear as anomalous points in the graph.
Interactions between blocks and treatments may create the appearance of unequal variances between the treatments.
Even if none of the test assumptions are violated, a one-way blocked ANOVA with small sample sizes may not have sufficient power to detect any significant difference among the samples, even if the means are in fact different. The power depends on the error variance, the selected significance (alpha-) level of the test, and the sample size. Power decreases as the variance increases, decreases as the significance level is decreased (i.e., as the test is made more stringent), and increases as the sample size increases. With very small samples, even samples from populations with very different means may not produce a significant one-way blocked ANOVA F test statistic unless the sample variance is small. If a statistical significance test with small sample sizes produces a surprisingly non-significant P value, then a lack of power may be the reason. The best time to avoid such problems is in the design stage of an experiment, when appropriate minimum sample sizes can be determined, perhaps in consultation with a statistician, before data collection begins.
For one-way blocked ANOVA, Prophet uses the iterative imputation method described by Glen and Kramer (1958). Any imputation method can only deal successfully with data that have only a relatively small percentage of missing values, and preferably with only one missing value. The power of the F test is reduced because there are fewer actual data points than for a complete design, and the degrees of freedom must be adjusted accordingly, which tends to decrease the F statistic.
Back
Back