User`s guide

E-Prime User’s Guide
Appendix B: Considerations in Research
Page A-21
subjects required. In this case, a Latin square design can approximate complete
counterbalancing.) An alternative is to randomize the order of presentation of the experimental
conditions for each subject. Over a fairly large number of subjects, this will approximate
counterbalancing. Note that with either counterbalancing or randomization, recording the order of
the conditions in the data file will permit later comparison explicitly on the performance of subjects
receiving different orders of the experimental conditions.
20
Before Beginning …
Before beginning to design an experiment, carefully consider the broader research question that
is trying to be answered. While the use of computers with software such as E-Prime makes it
easier to run experiments, there is still a lot of cost involved in paying subjects, in time testing
subjects and analyzing data. For that reason, time spent “up front” on careful theoretical
considerations will avoid wasted effort and increase the chance of getting an interpretable and
publishable result. In this section, we consider a number of issues that need to be addressed
before and during the detailed process of experimental design.
What are the questions that need to be answered?
Before beginning to design an experiment, have a clear formulation of the questions trying to be
answered. Specify a hypothesis, or a statement about the expected effects of an independent
variable on the dependent variable (e.g., reaction time will decrease as the flanking letters are
moved farther from the target letter). The hypothesis may come from an explicit theory, may
represent an extension of previous research, or may come from personal observation. In
exploratory research, the questions may concern the nature of a phenomenon—what are the
conditions under which the phenomenon (e.g., a visual illusion) occurs? Here, the concern is not
with testing a theory, but with delineating a phenomenon. In confirmatory research, the
research questions concern the explicit test of a theory about the nature of a phenomenon. If the
experimental result is predicted in advance by the theory, that tends to confirm the theory.
However, if an experimental result contradicts a prediction of the theory, it suggests that the
theory is at least incomplete, and possibly incorrect. (A thorough discussion of the confirmation
and falsification of theories lies far beyond the scope of this chapter. See Elmes, Kantowitz, &
Roediger, 1992.)
How can the research questions be answered?
Whether the research is exploratory or confirmatory, the questions to be answered must be as
specific as possible, so that what would count as evidence is clear. It is important that the
questions be posed in such a manner that some kind of experimental task can be devised that
can answer the questions. Comparisons are at the heart of any scientific question—it is expected
that a dependent variable will, in fact, vary as the level of the independent variable(s) is changed.
In confirmatory research, there is a specific prediction of at least the direction (possibly the
degree) of the differences in DV’s as IV’s vary. For example, a theory might predict that RT
would increase as the intensity of some IV changes. In exploratory research, there is no precise
20
We realize that it is now fashionable to refer to the persons from whom we obtain data as “participants”
(Publication Manual of the American Psychological Association, 4
th
ed., 1994). We continue to use the term
“subject,” because the whole point of doing an experiment is that you, the experimenter, manipulate the
independent variable. It is precisely because the person has agreed to temporarily suspend control and let
you decide the level of the IV to which they will be exposed, or the order of the levels, that makes the study
an experiment. Of course, the subject may remove himself or herself from participation at any time, but for
as long as they participate, subjects have allowed you to subject them to the conditions you choose.
“Participant” suggests a level of free choice that is not a part of an experiment (beyond the freedom to
choose whether or not to participate and to withdraw).