Please read.. I attached 3 discussions- Exercise 5,6,7  All need to be done on separatePlease read.. I attached 3 discussions- Exercise 5,6,7  All need to be done on separate page with the this book as the reference.  
 

Edmonds, W. A., & Kennedy, T. D. (2017). An applied guide to research designs: Quantitative, qualitative, and mixed methods (2nd ed). Thousand Oaks, CA: Sage.

I also copy the chapter readings so you can have because this professor wrote this book and he knows if we are messing around with content. 

Exercise 5 – Non-experimental Research

Why would a researcher choose to conduct non-experimental research? What are the strengths and weaknesses associated with this type of research?

Can cause and effect be established via this type of research? Why or why not?

Exercise 6 – Validity and the Observational Approach

Identify the types of design validity that should be considered when utilizing the observational approach? Select one specific type (e.g., test sensitization) and provide an example detailing how one might avoid violating that particular type of validity.

Exercise 7 – Validity and the Survey Approach

No unread replies.No replies.

Identify the types of design validity that should be considered when utilizing the survey approach? Select one specific type (e.g., sample characteristics) and provide an example detailing how one might avoid violating that particular type of validity.

rt II Quantitative Methods for Nonexperimental Research

This part includes two popular approaches to the quantitative method (nonexperimental only), followed by some of the associated basic designs (accompanied by brief descriptions of published studies that use the design). Visit the companion website at 

study.sagepub.com/edmonds2e

 to access valuable instructor and student resources. These resources include PowerPoint slides, discussion questions, class activities, SAGE journal articles, web resources, and online data sets.

Nonexperimental research is conducted when the researcher does not have direct control of the independent variables simply because their manifestations have already occurred. That is, nonexperimental research is used when the variables of interest cannot be controlled though the means of manipulation, inclusion, exclusion, or group assignment. However, the one form of control that can be used for nonexperimental research is through different types of statistical procedures. This form of control can include the matching or pairing of alternate forms of independent variables (e.g., gender, socioeconomic status) prior to conducting an analysis or through the application of regression-discontinuity approaches to existing data sets, causal-modeling techniques, or propensity-scoring methods (i.e., equating groups on known covariates and assigning to conditions). Although the research is considered nonexperimental, it does not imply that less value or meaning may be derived from the findings. Typically, causal relationships can only be inferred from experimental research; but considering the limitations and difficulty applying experimental research, findings from nonexperimental research (observational/correlational) can be a compelling indicator of cause and effect (e.g., the relationship between smoking cigarettes and lung cancer).

Much of our knowledge today regarding cause-and-effect relationships (from astronomy to epidemiology) is derived from the nonexperimental research of observational data. As previously noted, researchers believe that causal links can be drawn from examinations that are nonexperimental in nature, particularly with the application of causal-modeling techniques and propensity-scoring methods. The reader is referred to Rubin (2007, 2008) for more information regarding the approximation of observational approaches to experimental tenets, as well as Imbens and Rubin (2015) for statistical procedures designed to infer causal inference from observational data.

Again, in nonexperimental research, the researcher does not attempt to control or manipulate the actual conditions (i.e., independent variables); rather, control is exerted over levels of variables (through statistical procedures). Hence, technically speaking, the concept of internal validity does not apply to nonexperimental research. Nonexperimental research is primarily used to explain or predict relationships or to describe and measure the degree of association (relationship) among variables. However, unlike internal validity, issues related to external, construct, statistical conclusion validity should still be accounted for when conducting nonexperimental research, although these concepts were originally defined to address the issues related to the determination of cause-effect relations through experimental or quasi-experimental research.

We present a one-group posttest-only design, which is considered nonexperimental and one of the “weakest” designs presented in this book. We also present one example of an ex post facto design. However, the two most common forms of quantitative methods and nonexperimental research are observational and survey approaches (sometimes referred to in texts as correlational designs and descriptive research, respectively), which are presented in this part. As a simple reminder, the critical difference between experimental and nonexperimental research is the concept of control of the independent variable(s).

Chapter 8 Ex Post Facto and Posttest-Only Designs

Another form of nonexperimental research is called the ex post facto (after the fact) design, sometimes referred to as causal-comparative research (the term causal-comparative is considered a misnomer). This widely misunderstood and underused design is an attempt at creating quasi-experimental research out of nonexperimental research. Ex post facto designs are used when the researcher cannot control the treatment variable (i.e., the treatment and control groups are selected after the treatment has occurred), and there are no pretest measures, while only a posttest is collected. Unlike all the designs in nonexperimental research, the ex post facto design is unique in that issues related to internal validity still should be considered when evaluating the outcomes. The major threats include history, selection bias, maturation, and attrition. Clearly, the most obvious threat is selection bias because groups are self-selected and nonrandomly assigned to conditions for a multitude of reasons. Therefore, researchers can implement some type of “control” for the selection-bias issue by using a post hoc matched-grouping technique. This allows the researcher to establish control over the variables of interest—that is, because the independent (treatment) variable is not manipulated, various levels of alternate independent variables (e.g., age or gender) can be statistically manipulated (controlled) and used as a means to include individuals in the desired conditions. These alternate independent variables are sometimes referred to as quasi-independent variables because they are not subjected to the various control techniques (e.g., manipulation, elimination). See 

Diagram 8.1

 for an example of an ex post facto design. The reader is referred to Giuffre (1997) and Spector (1981) for more details regarding ex post facto designs.

Diagram 8.1 Two-Group Ex Post Facto Design

Note: MA represents the matched-grouping criteria (i.e., statistical procedures) used as a means to include the desired participants in each condition. The assignment to conditions is conducted after the treatment has occurred.

The one-group posttest-only design (often referred to as the one-shot case study) is considered nonexperimental and a “weak” design. Although the one-group posttest-only design is nonexperimental, threats to internal validity still should be a consideration—that is, the major threats to internal validity associated with this design are what determine the limitations in assessing the outcome. The obvious threats are selection bias and special treatment. Because there is only one designated observation with no comparison groups or multiple observations within subjects, it is nearly impossible to rule out plausible alternative explanations (i.e., the identified cause cannot be determined to be the only explanation for the effect).

Diagram 8.2 Posttest Design (One-Group)

Note: Statistical procedures is the only form of control to be used in nonexperimental research. However, this design is unique in that the independent variable can also be controlled via elimination and manipulation. This is the only exception in nonexperimental research.

Example for Diagram 8.2

Morgan, B. J. (2001). Evaluation of an educational intervention for military tobacco users. Military Medicine, 166(12), 1094–2001.

Research Question: What is the short-term effect of a tobacco-hazard education intervention on tobacco use and intent to quit?

Procedures: A tobacco-hazard education intervention was developed and presented to military tobacco users. The presentation lasted approximately 1 hour with a follow-up question-and-answer period. One month after the intervention, participants were asked to complete a survey regarding tobacco use and their intent to quit.

Design: Nonexperimental research using a one-group posttest-only design

Recommended Statistical Analysis: Descriptive statistics

, one-sample t test

Diagram 8.3 Ex Post Facto Design

Note: The current example included one treatment for both groups, but some applications of this design can include two separate treatments or a treatment and control.

Example for Diagram 8.3

Chapin, M. H., & Holbert, D. (2009). Differences in affect, life satisfaction, and depression between successfully and unsuccessfully rehabilitated persons with spinal cord injuries. Rehabilitation Counseling Bulletin, 53(1), 6–15.

Chapter 9 Observational Approach

The observational approach is considered a correlational approach to research. The researcher does not intervene or use experimental control (i.e., manipulation, elimination, inclusion, group or condition assignment), hence this is considered nonexperimental research. Investigators use this approach when they are interested in measuring the degree of association (i.e., relationship) between variables or to predict some outcome (criterion) based on the predictor variable(s). The only type of control that can be used for nonexperimental research and the observational approach is statistical procedures. Because statistical techniques are the only form of control to be applied to the observational approach, some researchers and authors tend to refer to the actual analysis as the research design. This is misleading and not accurate because the statistical analysis is a tool to be applied to summarize the data which is organized via the research questions and research design.

Using correlational analyses or regression analyses, researchers can measure the strength (magnitude) and direction of the relationship between variables or predict the influence one variable has on another. Researchers can also apply analysis to observational data that compares the mean differences of multiple groups. Some researchers prefer to use the word explanation instead of prediction (i.e., if a phenomenon can be explained, then it can be predicted, although a prediction does not infer explanation). As previously noted, results from nonexperimental observational data oftentimes can provide a strong case for making causal inferences (e.g., the systematic observation of many data points over time, indicating that texting while driving increases the likelihood of getting into an accident). However, scientists should be cautious when making causal inferences based on nonexperimental observational data. Sample size and correlational inference is an important aspect to consider for the observational approach (see Anderson, Doherty, & Friedrich, 2008). The two most common designs within the observational approach are explanatory and predictive designs.

Explanatory Design

Through correlational or regression analysis, investigators attempt to explain the degree of association between two (or more) variables (sometimes referred to as relational research). Analyses can also be used to compare means of multiple groups depending on the research objectives (e.g., ANOVA, t test).When applying this design, data are collected at one point in time (theoretically) from a single group. Data can also be collected from extant data sets (i.e., retrospective analysis); once the research questions are applied to the data, and the design is identified, the appropriate analyses can be employed. More advanced explanatory designs can include collecting data on multiple variables as a means to confirm the direct and indirect effects between the variables (i.e., researchers attempt to infer causation through the application of causal modeling and confirmatory factor analysis and are sometimes referred to as single-stage and multistage models). Various forms of regression such as multiple, canonical, and cluster analysis, along with structural equation modeling, are used to summarize the data from these advanced explanatory designs (see Kline, 2010, for more information on multistage models).

Predictive Design

The predictive design goes beyond the explanatory design in that it allows the researcher to anticipate or predict the outcome based on the analysis of the relationship between two or more variables. Within this design, at least one variable is indicated as the predictor variable, and one variable is designated as the criterion or outcome variable. Advanced predictive designs can include multiple predictor variables, while requiring more advanced forms of regression analyses to summarize the data. Time is a factor built into this design, so the researcher will typically collect data on the predictor variable(s) at one point and then at a later point collect data on the criterion variable(s). Furthermore, data can be collected from extant data sets and the predictive design can thus be applied to set up the appropriate research questions and subsequent analyses, as long as the concept of time appropriately exists between the predictor and criterion variables.

Statistical Tools

Some statistical tools and example research questions follow that would warrant the use of the particular tool. These analyses can be applied to observational data, depending on the research objective, when there is no independent variable (IV) to be manipulated. These are simplified and general guidelines. As a reminder, various sources refer to IVs differently between various forms of research (refer to 

Chapter 1

 for a further discussion on this topic). We will reserve the use of the term IV for experimental or quasi-experimental research, and for nonexperimental research, we will refer to the IV as the predictor variable and the outcome or dependent variable as the criterionvariable.

· Comparison of Means for the Explanatory Design (one criterion variable, otherwise known as univariate comparison of means)

· t Test (independent samples). What are the differences between School A and School B on standardized test scores? The test score is the criterion variable.

· ANOVA. What are the differences between Schools A, B, and C on standardized test scores?

· ANCOVA. What are the differences between Schools A, B, and C on standardized test scores controlling for socioeconomic status (SES)? SES is considered the covariate.

· Trend Analysis. What are the quality-of-life-ratings of working college students for freshmen, sophomores, juniors, and seniors? These students had jobs as they entered college and maintained work throughout all 4 years of their college span.

· 2-Way ANOVA. What are the differences between gender and SES on standardized test scores? The two predictors (IVs) used are gender (male and female) and SES (high and low), both broken down into two levels each.

Predicting a Single Variable for the Predictive Design

· Hierarchical Linear Regression. What is the level (or prediction) of perceived satisfaction of voters based on years of voting experience, hours devoted to studying candidates, and ideas of collectivism as reported by political science professors? The researchers are interested in predicting one aspect of satisfaction (criterion variable), which is broken down by three subscales in the assessment: individual, collective, and familial. The predictor variables are voting experience, number of hours devoted to studying candidates, and collectivism.

· Multiple Regression. How do levels of exercise and mental training predict levels of HbA1c for patients with type 2 diabetes? The interaction between exercise and mental training will also be examined.

· Mediation. To what extent is the predictive effect of exercise on levels of HbA1c for type 2 diabetics mediated by mental training?

Predicting More Than One Variable

· Binary Logistic Regression. To what extent do gender, age, and entrance exam scores predict graduate school success? Success is the binary criterion variable and is conceptualized as successful or not successful.

· Multinomial Logistic Regression. To what extent do narcissism, self-efficacy, and gender predict type of leadership style (autocratic, democratic, equal)? The criterion variable in this example (leadership style) is broken down into three levels: autocratic, democratic, equal.

There are many other forms and types of analyses that can be applied to observational data sets. Keep in mind that statistical guides and texts often refer to the variables as either predictor variables or independent variables, and for the criterion variable, it is often referred to as the outcome or dependent variable. These analyses can be used for the explanatory or predictive designs based on the research question(s). Examples of other analysis that can be used, depending on access and type of data, are multilevel linear modeling, contextual modeling, mixed models, discriminant function analysis, and principal and exploratory component analysis.

Diagram 9.1 Explanatory Design

We refer the reader to the following books for further details regarding observational approaches and analyses:

· Imbens, G. W., & Rubin, D. B. (2015). Causal inference for statistics, social, and biomedical sciences: An introduction. Cambridge, UK: Cambridge University Press.

· Keith, T. Z. (2014). Multiple regression and beyond (2nd ed.). Boston, MA: Pearson.

· Meyers, L. S., Gamst, G., & Guarino, A. J. (2012). Applied multivariate research: Design and interpretation (2nd ed.). Thousand Oaks, CA: Sage.

Example for Diagram 9.1

Walker, C. O., & Greene, B. A. (2009). The relations between student motivational beliefs and cognitive engagement in high school. Journal of Educational Research, 102(6), 463–471.

Research Question: What is the association between classroom motivation variables and students’ sense of belonging?

Procedures: Questionnaires were distributed to participants (N = 249) during the middle of the term and completed in English classes. Perceived instrumentality and self-efficacy items were taken from the Approaches to Learning Survey. Student and classroom-achievement goal orientations were measured using the Patterns of Adaptive Learning Survey. Sense of belonging was assessed using the Psychological Sense of School Membership scale.

Design: Nonexperimental research using an observational approach with an explanatory design

Recommended Statistical Analysis: Correlational analysis

Diagram 9.2 Predictive Design1

Note: A designated period of time must elapse before data on the criterion are collected.

Example for Diagram 9.2

Erdogan, Y., Aydin, E., & Kabaca, T. (2008). Exploring the psychological predictors of programming achievement. Journal of Instructional Psychology, 35(3), 264–270.

Research Question: What mental factors significantly predict programming achievement?

Procedures: Forty-eight students (N = 48) completed four different measurement tools that served as the predictor variables: (a) KAI creativity scale, (b) Problem Solving Inventory (PSI), (c) General Skills Test Battery (GSTB), and (d) Computer Attitude Scale (CAS). The criterion variable was measured using the Programming Achievement Test (PAT) after students completed the programming language course designed to introduce them to basic concepts of structured programming.

Design: Nonexperimental research using an observational approach with a predictive design

Recommended Parametric Analysis: Regression analysis

or discriminant analysis

Example for Diagram 9.3

Hinnant, J. B., O’Brien, M., & Ghazarian, S. R. (2009). The longitudinal relations of teacher expectations to achievement in early school years. Journal of Educational Psychology, 101(3), 662–670.

Research Question: To what extent do factors of teacher expectations in the early school years predict future academic performance?

Procedures: Children from 10 sites were followed from first to fifth grade (N = 2,892). Two measures of children’s academic abilities were collected in the spring of the children’s first- (n = 966), third- (n = 971), and fifth-grade (n = 955) years: (a) teacher reports of classroom performance in reading and math and (b) children’s scores on standardized measures. In the spring of the first, third, and fifth grades, children were administered two subtests from a standardized psychoeducational assessment. A discrepancy score between teacher report of child academic performance and children’s observed performance on standardized tests was calculated to determine the congruency of teacher expectancy and academic performance.

Design: Nonexperimental research using an observational approach with a predictive design
Recommended Parametric Analysis: Regression analysis

Chapter 10 Survey Approach

The most common form of nonexperimental research is the survey approach (sometimes referred to as descriptive research). Typically, investigators administer a survey to a randomly selected sample of individuals or, if possible, to an entire population (see Fowler, 2013). Random selection is a critical element to survey research in that generalization (external validity) is the primary goal of the findings (i.e., external validity is the focus; internal validity does not apply). However, it must be noted that construct validity and statistical conclusion validity do apply to the survey approach. Again, the concept of internal validity is concerned with the establishment of cause-effect relations, whereas the survey approach is not applied to determine cause and effect. Surveys are used to observe trends, attitudes, or opinions of the population of interest. Participants are usually selected from the population to discover the relative incidence, distribution, and interrelations of educational, sociological, behavioral, or psychological variables. Thus, it can be classified as quantitative and is often considered a variant of the observational approach.

Often, as noted, when applying the survey approach, the goal is to eventually generalize the findings to the entire population. To achieve this, some form of a probability sampling strategy should be employed when applying the survey approach. Many major news outlets conduct surveys on a regular basis, but they are considered nonscientific, and the findings are not expected to generalize to the greater population. Why? First, the scientific method was not applied to the scenario, and second, a nonprobability sampling strategy was used to collect the data (only viewers of that particular news source and users of the specific website participate).

Primary Threats to Validity

External validity. Keep in mind that external validity, by definition, is the construct that is related to generalization. The major threat to external validity for the survey approach is sample characteristics. Sample characteristics are the extent to which the sample surveyed represents the identified population. To ensure this form of validity is not violated (assuming generalization is desired), then the appropriate probability sampling strategy should be employed. The following are the various types of probability sampling: simple random, cluster, stratified, systematic, and multistage.

Construct validity. The focus for the survey approach is seated in measurement. Therefore, it is vital that one can generalize the findings or results of a survey to the theory and primary inquiry posed by the researcher. Issues related to the survey instrument directly affect the validity of the outcome. The primary threats to construct validity for the survey approach are reactivity to assessment or acquiescence response bias and timing of measurement. Typically when participants know they are being surveyed, they may change or alter the way they respond to items on a survey, which is different than the way they truly feel. This is known as reactivity to assessment or more commonly known as social desirability. The second major threat is the timing of the measurement. The time in which the survey is administered can greatly affect the results due to numerous conditions outside the control of the researcher.

Statistical conclusion validity. Surveys and assessment tools should measure and gather the information that the instrument is purported to measure. This is fundamental to any form of research. And the greatest threat to statistical conclusion validity for the survey approach is unreliability of the measures. Clearly, if the measure is not reliable or suffers from inadequate levels of validity, then the result or outcome will be compromised.

Response rate. Response rates for the survey approach are always notoriously low. Often, researchers can expect a 15% to 20% return rate for external surveys, and internal surveys (workplace surveys) may exceed those percentages. Researchers are often limited in time and resources (e.g., money for incentives), which adds to the problem of securing adequate response rates. The quality of the response rate can directly affect the validity of the outcome. Fincham (2008) recommended a 60% response rate should be the general goal for most types of research, but researchers should strive for at least 80% for the survey approach when the intent is to generalize to the entire population. Again, these numbers are often unrealistic, particularly for students conducting research for theses and dissertations. Mundia (2011) exemplified the threats to validity when social desirability and response rates are not secured, which should be noted in the potential limitations for all studies classified under the survey approach. As a result, in order to address “low” response rates, researchers should conduct nonresponse bias analysis. Drechsler (2015) and Thompson and Oliver (2012) offered recommendations for strategies to address nonresponse bias in the form of multiple and fixed-affect imputations.

The reader is referred to Lavrakas (2009) for an in-depth and comprehensive coverage of the survey approach and the open-access journal Survey Methodology (

http://www5.statcan.gc.ca/olc-cel/olc.action?objId=12-001-X&objType=2&lang=en&limit=0

).

Cross-Sectional Design

The cross-sectional design allows the researcher to collect data at one point in time. This design is one of the most common designs that media outlets use to present information of public opinion on political or social circumstances. The most common application of this design is to gather opinions or attitudes from one specific group. However, in many cases, the same instrument can be administered to different populations as a means to compare a group’s attitudes or opinions on the same variable. Basic descriptive statistical analyses are typically used to summarize data.

Longitudinal Design

An extension of the cross-sectional design is the longitudinal design. This design allows the researcher to collect survey data over a designated period of time with the same or different samples within a population. Researchers can collect data using trend (identify a population to examine changes over time), cohort (identify a subpopulation based on specific characteristics), or panel (survey the same people over time) studies. Based on theoretical and logistical considerations, longitudinal designs can include any combination of data collected from cohorts, trends, and panels. Variations of this design include the cohort-sequential design and the accelerated longitudinal design. These designs allow the researcher to collect data fr

 related cohorts over time to determine the extent of the relation between the cohorts (see Prinzie & Onghena, 2005).

Diagram 10.1 Cross-Sectional Design

We refer the reader to the following book for further details regarding survey approaches:

· Fink, A. G. (2012). How to conduct surveys: A step-by-step guide (5th ed.). Thousand Oaks, CA: Sage.

Example for Diagram 10.1

Jones, M. A., Stratten, G., Reilly, T., & Unnithan, V. B. (2004). A school-based survey of recurrent non-specific low-back pain prevalence and consequences in children. Health Education Research, 19(3), 284–289.

Research Question: What evidence exists to demonstrate the prevalence and consequences of recurrent low-back pain in children?

Procedures: Questionnaires were issued to seven different schools (N = 500). A cross-sectional sample of 500 participants, boys (n = 249) and girls (n = 251), was collected. Participants were required to complete a questionnaire to assess their low-back pain history. The questionnaire was designed to identify lifetime prevalence, point prevalence, recurrent prevalence, and duration of the low-back pain.

Diagram 10.2 Longitudinal Design

Note: Any number of variables and observations can be associated with this design.

Design: Nonexperimental research using a survey approach with a cross-sectional design

Recommended Statistical Analysis: Descriptive statistics

Example for Diagram 10.2

Steinfield, C., Ellison, N. B., & Lampe, C. (2008). Social capital, self-esteem, and use of online social network sites: A longitudinal analysis. Journal of Applied Developmental Psychology, 29, 434–445.

Research Questions: How does the use of a social network among a college population change over time? What is the directionality of the relationship between social network use and development of bridging social capital? How does an individual’s psychological well-being influence the relationship between social capital and social network site use?

Procedures: Survey data were collected from university students at two time points a year for 2 consecutive years. Initially, undergraduate students were sent an e-mail invitation with a short description of the study, information about confidentiality, an incentive for participation, and a link to the survey. Participants were surveyed on general Internet use, social network use, psychological well-being, self-esteem, and satisfaction. As a follow-up to the first-year survey, in-depth interviews were conducted with 18 students primarily drawn from the initial sample.