OUTLINE FOR CHAPTER
4 IN THE 6th EDITION
Reliability and Validity
- The Meaning of Reliability and the Reliability Coefficient
- A good synonym for "reliability"
- Three possible forms of the "reliability question"
- The continuum of possible values for the reliability coefficient
- Different Approaches to Reliability
- Test-retest and the coefficient of stability
- Parallel-forms/alternate-forms/equivalent forms and the coefficient
- Internal consistency:
- Split-half (regular and Guttman)
- KR20 and KR21
- Cronbach's alpha
- Interrater reliability:
- Percent agreement
- Pearson's r
- Kendall's coefficient of concordance
- Cohen's kappa
- Intraclass correlation (ICC)
- The Standard Error of Measurement
- The abbreviation, SEM
- Using the SEM to build "confidence bands"
- Reliability, SEM, and the metric of the test
- Warnings About Reliability
- Different reliability procedures conceptualize "consistency" differently
- Reliability coefficients say something about scores, not the actual
- Reliability coefficients only estimate reliability
- With tests having tight time limits, internal consistency coefficients
are too high
- Reliability is not the only criterion that should be used when
evaluating test data
- The Meaning of Validity and Its Relationship to Reliability
- A good synonym for "validity"
- High validity implies high reliability . . . but the reverse is
- Different Kinds of Validity
- Content validity
- Criterion-related validity:
- Construct validity:
- Convergent and discriminant validity
- The "known-groups" approach
- Factor analysis
- Warnings About Validity Claims
- Validity is different than reliability; hence, reliability does
not establish validity
- Validity is tied to the scores produced by an instrument, not
the instrument itself
- Efforts at assessing content validity require competent "judges"
- Efforts at assessing criterion-related validity require a high-quality
- Validity often relies on correlation . . . so remember the warnings
of Chapter 3
- Three Final Comments
- How high should reliability and validity be?
- Using multiple methods to assess instrument quality
- Does the presence of highly reliable and valid data guarantee
a good study?