Qualitative generalizations from quantitative analyses

Abstract

This paper considers the circumstances under which qualitative generalizations from statistical tests are possible. When observations from different conditions are assumed to be identically distributed except for a location parameter, then any difference that is established between the conditions will be entirely due to location. Furthermore, the direction of these differences will be independent of the statistic used, the scale of measurement or the shape of the distributions. This implies that significant differences between conditions generalize to latent variables (variables which are monotonie transformations of observed variables). Obtaining a significant difference between two conditions establishes the direction of that difference and obtaining significance in a test of differences between more than two conditions establishes a nonparametric correlation between the observed and population means. To make equivalent generalizations from main effects in factorial analyses of variance it must be assumed that the observations are random with respect to the other factors in the design. Correlation and regression analyses generalize to latent variables where all the variables are monotonically related. However, this implies that there is only one ordinally distinct latent variable. Where non‐monotonic relationships are possible and one distribution is treated as fixed, significant correlation and regression analyses do not generalize to latent variables. It is argued that while linear modelling may be an appropriate method of making quantitative predictions from a set of random variables, tests of differences are more interpretable than correlations where the aim is to study theoretically interesting latent variables.