ARTICLE CRITIQUE

INTRODUCTION

Problem

Is there a statement of the problem or a qualitative

Yes

No

N/A

topic of study?

Does the problem of topic indicate a particular focus of study?

Is the problem “researchable,” that is, can it be investigated

through the collection and analysis of data?

Is the background information on the problem presented?

Is the educational significance of the problem

discussed?

Does the quantitative problem statement indicate the variables of interest and the specific relationship between

those variables that were investigated?

Does the qualitative problem statement provide a general

indication of the research topic or issue?

When necessary, are variables directly or operationally

defined?

Does the researcher have the knowledge and skill to carry

out the proposed research?

Review of the Related Literature

Is the review comprehensive?

Are all references cited relevant to the problem under investigation?

Are most of the sources primary; i.e., are there only a few

or no secondary sources?

Have the references been analyzed and critiqued, and the results of various studies compared and contrasted? That

is, is the review more than a series of abstracts or annotations?

Is the relevancy of each reference explained?

Is the review well organized?

Does it logically flow in such a way that the references least

related to the problem are discussed first and the most

related references are discussed last?

Does it educate the reader about the problem or topic?

Does the review conclude with a summary and interpretation

of the literature and its implications for the problem investigated?

Do the implications discussed form an empirical or theoretical

rationale for the hypotheses that follow?

Are references cited completely and accurately?

Hypothesis

Are specific questions to be answered listed or specific

hypotheses to be tested stated?

Does each hypothesis state an expected relationship or

difference?

If necessary, are variables directly or operationally defined?

Is each hypothesis testable?

METHODS

Participants

Are the size and major characteristics of the population

studied

described?

Are the accessible and target populations described?

If a sample was selected, is the method of selecting the

sample clearly described?

Does the method of sample selection suggest any limitations

or biases in the sample?

For example, is stratified sampling

used to obtain sample subgroups?

Are the size and major characteristics of the sample described?

Does the sample size meet the suggested guideline for minimum

sample size appropriate for quantitative analyses?

Instruments

Do instruments and their administration meet guidelines for

protecting human subjects?

Have needed permissions been obtained?

Is the rationale given for the selection of the instruments (or

measurements) used?

Is each instrument described in terms of purpose, content,

validity, and reliability?

Are the instruments appropriate for measuring the intended

variables?

Does the researcher have the needed skills or experience to

construct or administer an instrument?

Is evidence presented to indicate that the instruments are appropriate for the intended sample? (For example, is the

reading level of an instrument suitable for sample participants?)

If appropriate, are subtests reliabilities given?

If an instrument was developed specifically for the study,

are the procedures involved in its development and validation

If an instrument was developed specifically for the study, are

administration, scoring or tabulating, and interpretation procedures fully described?

Is the correct type of instrument used for data collection?

(For example, using a norm-referenced instrument when a

criterion-referenced one is more suitable.)

Design and Procedures

Are the design and procedures appropriate for examining the research question or testing the hypotheses of the study?

Are the procedures described in sufficient detail to permit

them to be replicated by another researcher?

Do procedures logically relate to each other?

Are instruments and procedures applied correctly?

If a pilot study was conducted, are its execution and results

described as well as its impact on the subsequent study?

Are control procedures described?

Are theoretical and practical implications of the findings

discussed?

Are recommendations for future action made?

Are the suggestions for future action based on practical

significance or on statistical significance only; i.e., has the

author avoided confusing practical and statistical significance?

ABSTRACT OR SUMMARY

Is the problem restated?

Are the number and type of subjects and instruments described?

Is the design used identified?

Are procedures described?

Are the major results and conclusions restated?

Qualitative Research in General

Does the topic statement provide a general sense of the study

focus?

Is the purposive sampling procedure described and related to

the study focus?

Is each data collection strategy described?

Is the role the researcher assumed stated (e.g., observer,

participant observer, interviewer, etc.)?

Is the research site and the researcher’s entry into it

Were the data collection strategies used appropriately, given

the purpose of the study?

Were strategies used to strengthen the validity and reliability

of the data (e.g., triangulation)?

Is there a description of how any unexpected ethical issues

were handled?

Were strategies used to minimize observer bias and observer effect described?

Are the researcher’s reactions and notes differentiated from

descriptive field notes?

Are data coding strategies described and examples of coded data given?

Is the inductive logic applied to the data to produce results

stated in detail?

Are conclusions supported by data (e.g., are direct quotes

used to illustrate points made)?

Observation Studies

Are observational variables defined?

How were observers trained?

Did different observers work and score independently?

Were observers required to observe only one behavior at a

time?

Was a coded recording instrument used?

Are the qualifications and special training of the observers

Was the level of interobserver reliability sufficiently high?

Were efforts made to overcome observer bias and observer

effect?

Was observation of subjects the most appropriate approach

for data collection (as opposed to use of some unobtrusive measure)?

Was a description of how the observational data were

analyzed provided?

Historical Research

Were the sources of data related to the problem mostly

primary?

Was each piece of data subjected to external criticism?

Was each piece of data subjected to internal criticism?

Does the researcher examine the possibility of personal bias in

the study analysis and conclusion?

Are causal inferences or conclusions warranted given the data

studied?

Is the report of the study an integrated, synthesized,

chronological presentation of the results?

Survey — Questionnaire Studies

Are questionnaire validation procedures described?

Was the questionnaire pretested?

Are pilot study procedures and results described?

Are directions to questionnaire respondents clear?

Does each item in the questionnaire relate to one of the

objectives of the study?

Does each questionnaire item deal with a single concept?

When necessary, is a point of reference given for questionnaire

items?

Does the cover letter explain the purpose and importance of the

study and give the potential responder a good reason for

cooperating?

If appropriate, is confidentiality or anonymity of responses

assured in the cover letter?

What is the percentage of returns and how does it affect the

study results?

Are followup activities to increase returns described?

If the response rate was low, was any attempt made to determine

any major differences between responders and nonresponders?

Are data analyzed in groups or clusters rather than a series

of many single variable analyses?

Survey — Interview Studies

Were the interview procedures pretested?

Does each item in the interview guide relate to a specific

objective of the study?

When necessary, is a point of reference given in the guide for

interview items?

Are leading questions avoided in the interview guide?

Is the language and complexity of the questions appropriate

for the participants?

Does the interview guide indicate the type and amount of

prompting and probing that was permitted?

Are the qualifications and special training of the interviewers

Is the method used to record responses described?

Did the researcher use the most reliable, unbiased method of

recording responses that could have been used?

Did the researcher specify how the responses to semistructured and unstructured items were quantified and analyzed?

Correlation — Relationship Studies

Were variables carefully selected; i.e., was a shotgun approach avoided?

Is the rationale for variable selection described?

Are conclusions and recommendations based on values of correlation coefficients corrected for attenuation or restriction in range?

Do the conclusions avoid suggesting causal relationships between the variables investigated?

Correlation — Prediction Studies

Is a rationale given for selection of predictor variables?

Is the criterion variable well defined?

Was the resulting prediction equation validated with at least one other group?

Causal — Comparative Studies

Are the characteristics or experiences that differentiate the groups (the independent variable) clearly defined or described?

Are critical extraneous variables identified?

Were any control procedures applied to equate the groups on extraneous variables?

Are causal relationships found discussed with due caution?

Are plausible alternative hypotheses discussed?

Experimental Research

Was an appropriate experimental design selected?

Is a rationale for design selection given?

Are sources of invalidity associated with the design identified and discussed?

Is the method of group formation described?

Was the experimental group formed in the same way as the control group?