Discuss about the characteristics and requirements that research tools should have on quantitative research to give validity to the study. Give some examples
Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins
Don't use plagiarized sources. Get Your Custom Essay on
Data Collection in Quantitative Research & Measurements and Data Quality
Just from $13/Page
Chapter 13
Data Collection in Quantitative Research
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question
Tell whether the following statement is true or false:
For unstructured data, researchers use formal data collection instruments that place constraints on those collecting data and those providing them.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer
False
For structured data, not unstructured data, researchers use formal data collection instruments that place constraints on those collecting data and those providing them.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Data Collection Plan
Structured data:
Formal data collection instruments
Constraints
Collection
Providing
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Early Steps in Data Collection Plan
Identification prioritization of data needs
Measures of variables
Selection existing instruments
Conceptual stability
Data quality
Cost
Population appropriateness
Reputation
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question
Tell whether the following statement is true or false:
Open-ended questions permit respondents to reply in narrative fashion.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer
True
Open-ended questions permit respondents to reply in narrative fashion, whereas closed-ended (or fixed-alternative) questions offer response alternatives from which respondents must choose.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Structured Self-Report Instruments
Interview schedules
Questionnaires
Open-ended questions
Closed-ended questions
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question
Which type of closed-ended question have several questions with the same response format?
Forced choice
Rating
Checklist
Visual analog scale
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer
C
Forced-choice questions require respondents to choose between two competing positions, rating questions ask respondents to make judgments along a bipolar dimension, checklists have several questions with the same response format, and visual analog scales (VASs) are continually used to measure subjective experiences.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Closed-Ended Questions
Dichotomous
Multiple choice
Rank order
Forced choice
Rating
Checklists
Visual analog scales
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Likert Scale
Summated rating scale
Series of statements about a phenomenon
Indicate degree of agreement or disagreement
Total score is computed by summing item scores, each of which is scored for the intensity and direction of favorability.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Semantic Differentials
Bipolar rating scales
Indicate reactions toward a phenomenon
Measure:
Evaluative
Activity
Potency
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Q Sort
Set of card statements into piles
Specified criteria
Measure:
Attitudes
Personality
Psychological traits
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Vignettes
Brief descriptions of event
Asked to react to events
Assess respondents:
Perceptions
Hypothetical behaviors
Decisions
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question
Tell whether the following statement is true or false:
Interviews are less costly and time-consuming than questionnaires.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer
False
Questionnaires are less costly and time-consuming than interviews.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Questionnaires
Less costly
Less time-consuming
Anonymity
No risk of interviewer bias
Group administration
Most economical
Mail: low response rate
Web-based survey
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Interviews
Higher response rate
Wider variety of people
Richer data
Data quality
Interpersonal skills
Ease and build rapport
Probing
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Structured Self-Reports
Response set biases
Common responses
Social desirability
Extreme response
Acquiescence
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Checklists
Occurrence or frequency
Category systems
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Rating Scales
Rate phenomena
Dimension typically bipolar
Made at specific intervals or after observations are complete
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Sampling
Time sampling involves the specification of the duration and frequency of observational periods and intersession intervals.
Event sampling selects integral behaviors or events of a special type for observation.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Observer Biases
Enhancement of contrast effect
Central tendency bias
Halo effect
Assimilatory biases
Errors of leniency
Errors of severity
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Biophysiologic Measures
In vivo
In vitro
Objective
Accurate
Precise
Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins
Chapter 14
Measurement and Data Quality
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question
Tell whether the following statement is true or false:
Measurement involves assigning numbers to objects to represent the amount of an attribute.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer
True
Measurement involves assigning numbers to objects to represent the amount of an attribute, using a specified set of rules. Researchers strive to develop or use measurements whose rules are isomorphic with reality.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Measurement
The assignment of numbers to represent the amount of an attribute present in an object or person, using specific rules
Rules are necessary to promote consistency and interpretability.
Advantages
Removes guesswork
Provides precise information
Less vague than words
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Theories of Measurement
Psychometrics is a branch of psychology concerned with the theory and methods of psychological measurement.
Two theories
Classical test theory (CTT)
Item response theory (IRT)
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Errors of Measurement
Obtained Score = True score + Error
Obtained score: an actual data value for a participant
True score: value that would be obtained for a hypothetical perfect measure attribute
Error of measurement: represents measurement inaccuracies
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Factors That Contribute to Errors of Measurement
Situational contaminants
Transitory personal factors
Response-set biases
Administration variations
Instrument clarity
Item sampling
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Major Types of Measures
Generic
Specific
Static
Adaptive
Reflective scales
Formative indexes
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Measurement Taxonomy
Four measurement property domains
Cross-sectional domains
Reliability
Validity
Longitudinal measurement domains
Reliability of change scores
Responsiveness
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Correlation Coefficients
Correlation coefficients indicate direction and magnitude of relationships between variables.
Pearson’s r
Range:
From −1.00 (perfect negative correlation)
Through 0.00 (no correlation)
To +1.00 (perfect positive correlation)
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Reliability
Consistency—the absence of variation in measuring a stable attribute for an individual
Reliability assessments involve computing a reliability coefficient.
Most reliability coefficients are based on correlation coefficients.
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question
Tell whether the following statement is true or false:
Reliability coefficients usually range from .00 to 1.00, with higher values reflecting less reliability.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer
False
Reliability coefficients usually range from .00 to 1.00, with higher values reflecting greater reliability not less reliability.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Reliability
Replication approaches
Test-retest reliability: administration of the same measure to the same people on two occasions
Interrater reliability: measurements by two or more observers or raters using the same instrument or measurements by the same observer or rater on two or more occasions
Parallel test reliability: measurements of the same attribute using alternate versions of the same instrument, with the same people
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Reliability Coefficient (R)
Represent the proportion of true variability to obtained variability:
R = VT
Vo
Should be at least .70; .80 preferable
Can be improved by making instrument longer (adding items)
Are lower in homogeneous than in heterogeneous samples
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Internal Consistency
The extent to which all the instrument’s items are measuring the same attribute
Evaluated by administering instrument on one occasion
Appropriate for most multi-item instruments
Most widely used evaluation method is the coefficient alpha.
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Measurement Error
Unless a reliability coefficient is 1.0 (virtually never happens), measurement error is present.
Used to estimate the range within which the true score lies
Standard error of measurement (SEM)
Limits of agreement (LOA)
Measurement error is routinely estimated for multi-item measures developed with item response theory (IRT) methods.
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Validity
The degree to which an instrument measures what it is supposed to measure (resilience)
Four aspects of validity
Face validity
Content validity
Criterion-related validity
Construct validity
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Face Validity
Refers to whether the instrument looks as though it is measuring the appropriate construct
Based on judgment, no objective criteria for assessment
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Content Validity
The degree to which an instrument has an appropriate sample of items for the construct being measured
Relevance
Comprehensiveness
Balance
Evaluated by expert evaluation, via the content validity index (CVI)
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Criterion Validity
The degree to which the instrument correlates with an external criterion or “gold standard”
Focal measures
Expense, efficiency, risk and discomfort, criterion unavailable, and prediction
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Criterion Validity (cont.)
Two types of criterion-related validity:
Predictive validity: the instrument’s ability to distinguish people whose performance differs on a future criterion
Concurrent validity: the instrument’s ability to distinguish individuals who differ on a present criterion
Specificity, sensitivity
Predictive values
Likelihood ratios
Receiver operating characteristic curve (ROC curve), area under the curve (AUC)
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Construct Validity
Concerned with the questions:
What is this instrument really measuring?
Does it adequately measure the construct of interest?
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Methods of Assessing Construct Validity
Hypothesis-testing validity
Convergent validity
Known-groups validity
Divergent validity (discriminant validity)
Multitrait–multimethod matrix method (MTMM)
Structural validity
Cross-cultural validity
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Measuring Change
Change score: represents the amount of change between two scores
Difference score: the difference between the randomized groups at posttest
Smallest detectable change (SDC): a change in scores that is beyond measurement error
Reliable change index (RCI): assesses the clinical significance of improvement during a psychotherapeutic intervention
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Responsiveness
The ability of a measure to detect change over time in a construct that has changed, commensurate with the amount of change that has occurred
Whether a change score is truly capturing a real change in the construct
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Question
Tell whether the following statement is true or false:
Reliability is the degree to which an instrument measures what it is supposed to measure.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Answer
False
Validity is the degree to which an instrument measures what it is supposed to measure. Reliability is the degree of consistency or accuracy with which an instrument measures an attribute.
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Critiquing Data Quality in Quantitative Studies
Can I trust the data in this study?
Are the measurements of key constructs reliable and valid, and are change scores reliable and responsive?
*
Copyright © 2017 Wolters Kluwer Health | Lippincott Williams & Wilkins
Psychometric Assessment
Gather evidence:
Validity
Reliability
Other assessment criteria
