Perfunctory analysis of variance in agronomy, and its consequences in experimental results interpretation

Analysis of variance (ANOVA) is based on two main assumptions, i.e., normality and homogeneity of the variances of the populations samples are collected from. In order to verify the correct application of ANOVA in agronomic research, we revised the two most recent years of two high ranked journals concerning agronomy: European Journal of Agronomy and Field Crops Research. The main issues considered were: presence of tests for normality and homogeneity of variance, and eventually the possibility of identifying problems due to analysis carried out on data not matching these assumptions and to incorrect applications of multiple comparisons. Forty-six percent of the reviewed papers uses ANOVA and, in 60% of these papers, assumptions are not verified at all (and frequently there are evidences that assumptions are not met), or there is a misuse of multiple comparison tests. We also pointed out that the more relevant risk of transmitting erroneous information to the scientific community comes from the use of wrong techniques for multiple comparisons, in particular when protected least significant difference (LSD) test is used. This was demonstrated through exemplifications carried out using Monte Carlo simulations that showed an unacceptable rate of type-III errors found with the protected LSD methods for means separation. We think this study could represent a useful warning on how to avoid misleading conclusions from agronomic experiments due to the incorrect application of classical statistical techniques (i.e., procedures not fully controlling the type-I error rate at experimentwise level).