User`s guide

E-Prime User’s Guide
Appendix B: Considerations in Research
Page A-32
The main thing to note about the example above is that a within-subjects design is clearly better,
if it is appropriate to use it, because it effectively increases sample size. But there are severe
limitations to its use as well. A within-subjects design works fine in this example because if the
experimenter tests subjects drunk, then tests them sober a few days later, she can be fairly sure
that the only systematic difference in the subjects is in whether or not they are sober. Similarly,
when comparing RT to two versus four stimuli, making a choice between two stimuli probably
does not have a later effect on making a choice between four stimuli (or vice-versa)—at least if
the trials were blocked. But in many situations making the assumption that there is no carry-over
from one condition to another is not justified. For example, to compare RT to naming
meaningless shapes following two different types of training, a between-subjects design is
needed because if a subject learns something by one method that learning cannot be “erased." If
subjects performed faster following the second round of learning, is it because that method of
learning is better? Or is the difference simply due to the added learning? Another situation in
which a between-subjects design is required is when the variable is "attached" to the person, and
cannot be experimentally manipulated. Variables of this kind include sex, race, ethnic
background, and religion.
In general, then, within-subjects designs are to be preferred if it is reasonable to assume that
there are no carry-over effects of one level of an independent variable on performance at other
levels of that independent variable. If that assumption is not reasonable, a between-subjects
design should be used. Note that this is similar to the issue of random order of trials versus
blocking by trial type—if encountering one level of a variable might induce a strategy that carries
over to another level, the levels should be blocked, when using a within-subjects design. If
blocking will not solve the problem, a between-subjects design will be necessary.
Another way of thinking about when to use a within-subjects design is to consider whether the
effect of the experimental “treatment” or manipulation wears off. If the passage of time will erase
the effects of the manipulation, a within-subjects design may be appropriate. An old joke
illustrates the point. A lady walking down the street saw a man lying drunk in the gutter. “Sir,”
she said, in obvious disgust, “You are drunk!” Opening one eye the man replied, “Yes, madam,
and you are ugly. And tomorrow, I shall be sober.” Some treatments wear off, and thus are
candidates for within-subject manipulation.
There are also some experiments that employ both within- and between-subjects independent
variables. These are usually referred to as mixed designs. For example, to compare the patterns
of RT's for males and females in the letter-identification experiment, sex would be added as
another independent variable, in addition to location and whether the letter size was adjusted to
compensate for distance from central vision. Location and adjustment would be within-subjects
variables. But sex (male vs. female) would be a between-subjects variable, since no subject
could be in both groups.
Other Considerations in RT Research
A number of other factors that must be considered in designing research employing RT as a
dependent variable are discussed below. Wickens (1992, Chapter 8) provides a more detailed
account of most of these same issues.
Speed-accuracy trade-off
In research employing RT as the dependent variable, the interest is usually in showing that RT
differs for different levels of the IV(s). A serious problem can arise, however, if the conditions
associated with faster RT also have higher error rates. Such a situation is called a speed-