Your Right To Be Liberal Or Conservative
As I pointed out near the outset of this chapter, the various post hoc procedures differ in terms of how liberal or conservative they are. Ideally, a researcher ought to choose among these procedures after considering the way they differ in terms of power and control of Type I errors. Realistically, however, the decision to use a particular test procedure is probably influenced most by what computer programs are available for doing the data analysis or by what procedure was emphasized in a specific textbook or by a specific instructor, or by habit.
Regardless of the stated (or unstated) reasons why the researcher chooses to use a particular test procedure, you are in full control of how you interpret the results presented in the research report. If a researcher uses a test procedure that is too liberal or too conservative for your taste, remember that you have the undisputed right to accept only a portion of the researcher's full set of conclusions. In the extreme case, you can, if you wish, reject the totality of what is "discovered" in a research study because the test procedure employed to make statistical inferences was far too liberal (or far too conservative) for your taste.
(From Chapter 12 in the 6th edition, p. 272-273)
Copyright © 2012
Schuyler W. Huck