Validity of Research Results

1. When a extraneous variable systematically varies with the independent variable and influences the dependent variable, it is called:

a. Another dependent variable

b. A confounding variable

c. A moderating variable

d. An unreliable variable

2. Which of the following statements is true?

a.  A statistical relationship is sufficient evidence to infer causality

b.  Temporal order of the cause and effect is not important in inferring causality

c.  A statistical relation of X and Y is insufficient evidence for inferring causality

d.  Temporal order of cause and effect variables and statistical relation are all that are

            needed to infer causality

3. A school district examines a program that uses mentors to help very poor readers improve their reading performance.  The children in the program are at the 4^th percentile at pretest.  At posttest they are around the 20^th percentile. While it is possible that the program made the difference, another reason for the change in scores could be:

a. History 

b. Regression artifact

c. Multiple-treatment interference

d. Differential selection

4. A group of researchers do a study where children from particular classrooms are assigned to treatment or control conditions.  After the study, the researcher finds out that the students in the control group are higher achievers than those in the experimental group.  He found no treatment effect.  The failure to find an effect may be due to:

a.       A treatment effect

b.      A testing effect

c.       A differential selection effect

d.   A maturation effect

5. A researcher examines a program looking at the effects of mentoring on poor readers' reading achievement. He looks at two different schools.  One serves as the control and the other the experimental group. Both schools had reading achievement that was around the 50^th percentile. During the time that the mentoring program is in place in the experimental group, a statewide reading initiative is started in randomly selected schools.  The experimental, but not the control school is involved in the initiative. At the end of the year, the experimental group does better than the control.  From the information presented above, a likely threat to the internal validity of the study is: 

a. Selection by mortality interaction

b. Mortality

c. Selection-history effect

d. Selection-maturation effect

6. Which type of validity refers to the degree to which you can infer that the relationship between two variables is causal?

a. Internal validity

b. Population validity

c. Ecological validity

d. Statistical conclusion validity

7. Which type of validity refers to the ability to infer that the independent and dependent variables are related ant that the measured strength of the relationship is accurate?

a. Internal validity

b. Population validity

c. Ecological validity

d. Statistical conclusion validity

8. An extraneous variable that systematically varies with the independent variable and also influences the dependent variable is known as a _______________.

a  Confounding variable
b. Third variable
c. Second variable
d. Both a and b are correct

9. The use of multiple observers to allow cross-checking of observations to make sure that the investigators agree with what took place is known as _______.

a. Interpretive validity

b. Researcher bias

c. Multiple operationalism

d. Investigator triangulation

10. _____________ is the lowest inference descriptor of all because it uses the participant’s own words.

a. Participant feedback

b. A verbatim

c. Data triangulation

d. Investigator triangulation

11. ___________ refers to physical or mental changes that may occur within individuals over time, such as aging, learning, boredom, hunger, and fatigue.

a.  Instrumentation

b.  History

c.  Maturation

d.  Testing

12. What type of validity refers to the extent to which the results of a study can be generalized across time?

a.  Ecological validity

b.  External validity

c.   Internal validity

d.   Temporal validity

13. Which of the following best describes interpretive validity?

a.  Factual accuracy of an account as reported by the researcher

b.  Accurately portraying the meanings given by the participants to what is being studied

c.  Degree to which a theoretical explanation fits the data

d.  Ability to generalize the study results across settings

14. Which of the following terms is a strategy where the researcher actively engages in critical self-reflection about his or her potential biases and predispositions.

a.  Experimenter effect

b.  Reactivity

c.  Investigator triangulation

d.  Reflexivity

15. Which of the following is not considered one of the criteria for inferring causality?

a. Evidence that the independent and dependent variables are related

b. Evidence that the relationship between the variables being investigated is not due to a

confounding extraneous variable

c. Evidence that changes in variable A occur before changes in variable B

d. The temporal ordering of the variables being investigated does not matter because a

relationship is all that is really needed

16. The use of multiple data sources to help understand a phenomenon is one strategy that is used to promote qualitative research validity. Which of the following terms describes this strategy?

a. Data matching

b. Pattern matching

c. Data triangulation

d. Data feedback

17. What may happen when different comparison groups experience a different history event?

a. History effect

b. Selection-history effect

c. Selection effect

d. Group effect

18. What is another term that refers to a confounding extraneous variable?

a. Last variable

b. First variable

c. Third variable

d. Fourth variable

19. Which of the following refers to any systematic change that occurs over time in the way in which the dependent variable is assessed?

a. Instrumentation

b. Maturation

c. Testing

d. Selection

20. Which of the following terms describes the ability to generalize from the sample of individuals on which a study was conducted to the larger target population of individuals and across different subpopulations within the larger target population?

a.  External validity

b.  Population validity

c.  Ecological validity

d.  Temporal validity

21. Which of the following is not a strategy used to promote qualitative research validity?

a.  Peer review

b.  Theory triangulation

c.  Extended fieldwork

d.  Random assignment

22. The use of several measures of a construct is called:

a. Multiple operationalism

b. Multiple construct measurement

c. Operationalism

d. Methods triangulation

23. A physical or mental change that occurs in participants over time that affects their performance on the dependent variable is called ________.

a. Instrumentation

b. Maturation

c. Regression

d. None of above

24. Attrition generally occurs in research where ____.

a. You do demographic research

b. The study fails

c. Some participants do not complete the study

d. The study is very brief

25. Differential attrition occurs when the people dropping out from one group are different from the others in their group or from the people in the comparison group.

a. True

b. False

26. Internal validity refers to which of the following?

a. The ability to infer that a casual relationship exists between 2 variables
b. The extent to which study results can be generalized to and across populations of persons,

settings, and times

c. The use of effective measurement instruments in the study

d. The ability to generalize the study results to individuals not included in the study

27. Which strategy used to promote qualitative research validity uses multiple research methods to study a phenomenon?
a. Data triangulation
b. Methods triangulation
c. Theory triangulation
d. Member checking

28. Which type of validity refers to the factual accuracy of an account as reported by the researcher?

a. Ecological validity

b. Temporal validity

c. Descriptive validity

d.  None of the above

29. Which of the following in not one of the key threats to internal validity?

a. Maturation

b. Instrumentation

c. Temporal change

d. History

30. This type of validity refers to the ability to generalize the results of a study across settings.

a. Temporal validity

b. Internal validity

c. Ecological validity

d. External validity

31. Which is not a direct threat to the internal validity of a research design?

a. History

b. Testing

c. Sampling error

d. Differential selection

32. Alteration in performance due to being aware that one is participating in a study is known as ______.

a. Operationalism

b. Reactivity

c. Temporal validity

d. Mortality

33. The idea that the more times a research finding is shown with different sets of people, the more confidence we can place in the finding and in generalizing beyond the original participants is known as ___________.
a. Naturalistic generalization
b. Methods generalization
c. Data triangulation
d. Replication logic

Answers:

1. b

2. c

3. b

4. c

5. c

6. a

7. d

8. d

9. d

10. b

11. c

12. d

13. b

14. d

15. d

16. c

17. b

18. c

19. a

20. b

21. d

22. a

23. b

24. c

25. a

26. a

27. b

28. c

29. c

30. c

31. c

32. b

33. d