User`s guide

E-Prime User’s Guide
Appendix B: Considerations in Research
Page A-20
the experiment that will be manipulated. The independent variables may be manipulated by
randomly assigning subjects to conditions (levels of the IV), or by applying each condition to each
subject, in a random or counterbalanced order. In discussing statistical analysis of experiments,
independent variables are also sometimes called factors. An experiment with more than one IV
is said to use a factorial design.
Controls
Confounding variables are aspects of the experimental situation that are correlated with the IV’s
that the experiment is intended to study, and that may be producing (or hiding) differences among
different levels of the IV’s. An example may help. Suppose that a researcher wishes to compare
two methods of teaching basic statistics. She teaches two sections of the course, and so decides
to teach her 8:00 am class by Method A and her 10:00 am class by Method B. Suppose she
finds that students who learned by Method B have significantly higher mean scores on the
common final exam than those taught by Method A. Can she conclude that Method B is better?
Hardly. Perhaps students are less alert in 8:00 classes than in 10:00 classes. However, suppose
that she finds that there is no difference between the classes on the final exam. Can she
conclude that the method of teaching doesn’t matter? Again, hardly. In this case, perhaps
Method A is actually superior, but the 8:00 class was only half awake, and the only reason they
did as well as those taught by Method B was that Method A was sufficiently better to overcome
the problem of inattentiveness. In this example, time of day is confounded with method of
teaching. (Confounding method of teaching with time of day is not the only problem with this
design. The lack of random assignment of students to classes is also a problem.)
Controls include any aspects of the experiment that are intended to remove the influence of
confounding variables. Controls are usually intended to remove variability caused by confounds,
by making them constants, not variables. In the example above, that would involve controlling
the time of day at which the classes were taught. Another example: In research involving visual
displays, be concerned about the effective size of the stimuli, which would vary if different
subjects sat at different distances from the computer monitor. In that case, a relevant control
would be to use a viewing hood or chin rest to make the distance from the screen the same for all
subjects. A third example is: make sure that equal numbers of males and females are in the
group of subjects tested at each level of the independent variable, if it is suspected that there
might be sex differences in performance. By having equal numbers of males and females, any
effect of sex would be the same for each level of the IV, and any differences in average
performance for the two levels would not be due to having more males in one group or more
females in another.
Note that in the case of assigning equal numbers of males and females to each level of an IV, sex
has actually been added as a blocking variable. If recording the subjects’ sex in the data file,
later analyze the data separately for males and females to explicitly check for sex differences.
Blocking variables should always be included as “independent variables” in a data file. An
advantage to matching groups on a blocking variable is that it serves to control that confound and
to permit examination of its influence.
Order effects are an important class of confounds, especially in experiments in which each
subject serves at each level of the IV. Here, experiencing one level of the IV may change
performance at another level. Examples would be when experience in one condition provides
practice that improves performance in another condition, or when experience of the first condition
induces a strategy that affects performance on the other. Two general solutions are available:
counterbalancing and randomization. Complete counterbalancing guarantees that each
condition precedes or follows each of the others equally often. (For experimental designs with
many levels of an IV, complete counterbalancing is usually not possible, due to the number of