The purpose of this unit is to explore the validity
and reliability of instruments. You will complete two reliability projects for
this unit. Please read all of the instructor notes on reliability
and validity,
"The quality of the instruments used in research is
very important, for the conclusions researchers draw are based on the information
they obtain using these instruments....[It is imperative that] the inferences
they draw, based on the data they collect, are valid. Validity
refers to the appropriateness, meaningfulness, and usefulness of the inferences
a researcher makes. Reliability refers to the consistency of scores or
answers from one administration or an instrument to another, and from one set
of items to another" (Fraenkel & Wallen, 2000, p. 169).
First, learn about reliability
and then learn about validity.
As a point of interest...
If a researcher were to develop an instrument to measure students' attitudes,
the following steps would be taken:
Beginning Steps in
Developing an Attitude Instrument
- Review literature in the domain which you wish to
measure (i.e., "computer attitudes").
- Develop a list of categories (subscales) that you
wish to sample from the domain. The domain may be "Computer Attitudes" and the
categories may be "ease of use of computers" and "usefulness in
education".
- Write 8 to 10 items/statements (operational
definitions) for each category (i.e., "Computers will help students learn material
faster.").
- Give the items to at least 5 experts for
classification (Content Validity). The panel of
experts will attempt to match the operational definitions with their appropriate
categories within the domain.
- Develop an instrument with the successfully
classified items. Use a Likert scale to design your instrument.
You will rewrite the items that were not successfully classified.
- Field test the instrument (6 to 10 people per item on
the instrument) with the populations for which the instrument is being developed.
- Run a factor analysis (exploratory) on the field test
responses. More advanced students may wish to do a confirmatory factory analysis.
- Name each factor (category) based on the items which
loaded on it (>.40)
- Review whether each item conceptually belongs with
its factor (subscale) and remove those which do not.
- Run Cronbach's Alpha Reliability for each
factor/category (subscale) to investigate internal consistency reliability.
- Modify and retest the instrument if necessary
(We want alpha>.70 for an attitude instrument).