© 2014 by Pearson Higher Education, Inc
Upper Saddle River, New Jersey 07458 • All Rights Reserved

Class Name,
Instructor Name
Date, Semester
Hagan, Research Methods in Criminal Justice and Criminology, 9/e
Chapter 5
Survey Research: Questionnaires
*

*

5
Some Guidelines for Questionnaire Construction
Survey Research: Questionnaires
Variables list
A list of variables being measured keyed to
the question number in the questionnaire which are designed to measure each variable
Dummy tables
Preliminary blank tables constructed prior to data-gathering that suggest the type of data needed, as well as the type of data analysis
*

© 2014 by Pearson Higher Education, Inc
Upper Saddle River, New Jersey 07458 • All Rights Reserved

5

*

5
Some Guidelines for Questionnaire Construction
Survey Research: Questionnaires
Questionable Wording:

Avoid biased or leading questions
Avoid double-barreled questions
Avoid asking questions in an objectionable manner

Avoid assuming prior information on the part
of the respondent
Avoid vague wording
Avoid asking more than you need to know

Avoid “response set” patterns by using reversal questions
*

5
Some Guidelines for Questionnaire Construction
Survey Research: Questionnaires
Pretest
A reconnaissance operation or exploratory testing of the instrument using subjects who are similar to the group to be studied
*

5
Organization of the Questionnaire
Survey Research: Questionnaires
Questionnaire
The order of questions may influence the willingness of subjects to respond to the survey.
A common error in surveys is to begin with the demographic items such as age, sex, and race.
*

5
Organization of the Questionnaire
Survey Research: Questionnaires
A Guaranteed Low-Response Questionnaire:
What is your name?
How old are you?
What is your sex?
How much money do you earn?
Do you cheat on your income tax?

How is your sex life?

When was the last time you committed a crime?
*

5
Mail Surveys
Survey Research: Questionnaires
Mail surveys are a popular means of conducting surveys that offers wide coverage at a minimum cost.
*

5
Survey Research: Questionnaires

Advantages of Mail Surveys
Disadvantages of Mail Surveys
No field staff, no transportation costs
No interviewers, no interviewer bias effects
Possible differences
between respondents and nonrespondents regarding the issue being investigated
Lack of uniformity in response
Offers respondents greater privacy
Slowness of response to follow-up attempts

Wide geographical samples at a reasonable cost, effort,
and time
Nonresponse
Offers respondents time to think out their responses, leading to more considered answers
Some respondents may
misinterpret the questions
Escalating costs if several follow-ups are required
*

5
The Tailored Design Method
Survey Research: Questionnaires
This involves tailoring the survey to the group
being studied in order to foster trust, increase
rewards, and decrease the cost of participation.
Particularly problematic is obtaining the participation of reluctant, busy professionals or elite policy-makers.
*

5
Self-Report Surveys
Survey Research: Questionnaires
Surveys in which subjects are asked to admit to the commission of various delinquent and/or criminal acts.
*

5
Survey Research: Questionnaires

Advantages of Self-Report Surveys
These methods correlate well
with other measures
Disadvantages of Self-Report Surveys
Inaccurate reporting
Poor or inconsistent instruments
Deficient research designs
vs.
*

*
Survey Research: Questionnaires
5

Advantages of Internet Surveys
Disadvantages of Internet Surveys
Faster and cheaper than phone surveys
Cheaper and more accurate than paper surveys
Limited to those with internet access and email accounts
The risk of coverage error has limited most email surveys to specialized populations
Results can be published online instantly for viewing by the respondent as a reward for participating
Some respondents may misinterpret questions

If research is limited to special populations, theoretical access of 100% is possible
Email has generally failed to produce comparable results to mail surveys
Lack of anonymity

5
Procedures in Internet Surveys
Survey Research: Questionnaires
Market research has found web-based research particularly useful and is fueling website design and better means of utilizing this strategy.

Krauss, Michael. “Research and the Web: Eyeballs or Smiles?” Marketing News 32 (December 7, 1998): 18–23.
*

5
Procedures in Internet Surveys
Survey Research: Questionnaires
Web surveys can use color photography, video clips, and other dramatic enhancements. Some helpful hints in Internet surveys include:

KISS (Keep It Simple, Stupid). Be as simple and straightforward as possible with questions.
Do not ask what they can’t answer. For example: “Did your toilet training as a child affect your sex life as an adult?”
Don’t sell. This is another way of saying avoid leading questions.

McCullough, Dick. “Market Research on the Web: Guidelines for Success.” Communication World 15 (October–November 1998): 29–36.
*

5
Procedures in Internet Surveys
Survey Research: Questionnaires
Ways of increasing response rates on Web surveys include the following:
Banner ads on Web sites
Email invitations
Phone invitations
Email panels

McCullough, Dick. “Market Research on the Web: Guidelines for Success.” Communication World 15 (October–November 1998): 33.
*

5
Procedures in Internet Surveys
Survey Research: Questionnaires
Keyboard entry
Inputting coded material via the keyboard
Coder monitoring
Double checking coders’ work for accuracy
Code sheet
A transfer sheet in which each box/cell is provided for coded material
Coding
Assignment of numbers to responses
Codebook
Guidebook for numerically classifying each question to be coded
*

© 2014 by Pearson Higher Education, Inc
Upper Saddle River, New Jersey 07458 • All Rights Reserved

It is most important to remember that surveys, for the most part, measure respondent attitude and not behavior.
Some possible ways of reducing nonresponse in mail surveys are follow-up, payment or altruistic appeals, attractive format, sponsorship, endorsements, personalization, shortened format, and good timing.
Coding is the assignment of numerical values to questionnaire items.
Self-report surveys involve asking (usually anonymous) respondents to admit to a variety of offenses they have committed in the past.
CHAPTER SUMMARY

*

© 2014 by Pearson Higher Education, Inc
Upper Saddle River, New Jersey 07458 • All Rights Reserved

Class Name,
Instructor Name
Date, Semester
Hagan, Research Methods in Criminal Justice and Criminology, 9/e
Chapter 6
Survey Research: Interviews and Telephone Surveys
*

*

Survey Research: Interviews and Telephone Surveys
6
Structured Interviews
Consist of check-off responses to questions that are factual and easily fit an expected pattern
Unstructured Interviews
Provide open-ended responses to questions
Depth Interview
Are more intensive or detailed interviews that are particularly useful in life histories or case studies
Types of Interviews
*

Survey Research: Interviews and Telephone Surveys
6
Structured Interviews
Consist of check-off responses to questions that are factual and easily fit an expected pattern
The interviewer should avoid soliciting additional
comments but, when they occur, record them verbatim.
The main disadvantage of closed-ended questions is that they generally elicit only limited response patterns.
The advantages are easy administration and data processing.

Types of Interviews
*

Survey Research: Interviews and Telephone Surveys
6
Unstructured Interviews
Provide open-ended response to questions
No predetermined response categories are provided.
Open-ended items may present a tabulation nightmare but provide the qualitative detail
and complexity of response that may be required, particularly if the subject of study
is little known.
Types of Interviews
*

Survey Research: Interviews and Telephone Surveys
6
Depth Interview
More intensive or detailed interviews that are particularly useful in life histories or case studies
The researcher has a general list of topics to be explored but exercises great discretion and flexibility in the manner, timing, and direction of questioning.
Types of Interviews
*

*
Survey Research: Interviews and Telephone Surveys
6

Advantages of Interviews
Disadvantages of Interviews
Personal contact between
researcher and subject
Interviewer can make use of cards, charts, & other AV aids
Time-consuming and costly
Interviewer effect/bias may distort results
Interviewer can pitch the language to respondent’s level
Interviewer may make errors in asking questions or recording information

Interviewer can use discretion as to the appropriate time to ask the more sensitive questions
Less convenient to the respondent, less anonymity than mail surveys

Survey Research: Interviews and Telephone Surveys
6
The Exit
Adminis-tration of the Interview
Session Arranging the Interview
Training and Orien-tation
*

Survey Research: Interviews and Telephone Surveys
6
Where possible, the field surveyors should match, as closely as possible, the subjects with respect to age, sex, race, social class, and dress. The interviewer’s language style should be adapted to the group studied.
Attire should be comfortable, but the interviewer should be neither overdressed nor underdressed.
Interviewers should have experienced a few practice interviews beforehand.
Interviewers should assure respondents that their responses will be held in strictest confidence.
Interviewers should attempt to build a rapport with the subjects.

Demeanor of Interviewer
*

Survey Research: Interviews and Telephone Surveys
6
Probing involves asking follow-up question(s) to focus, expand, clarify, or further explain the response given.
The interviewer should be familiar with the responses needed to each question to know when a probe is necessary.
The probe should not appear to be a cross-examination but should be a natural extension of the interview.
Probing
*

Survey Research: Interviews and Telephone Surveys
6
The interviewer should use parentheses to distinguish personal observations from the actual interview.
Editing entails reviewing the interview schedule after completion of the interview and cleaning it up and preparing it for analysis.
Unanswered questions should be marked “NA” for “not applicable” or simply “X” to indicate “inappropriate.”
Where personal observations are included, it may help to cross-reference relevant items.
Interviewers should avoid summarizing/paraphrasing responses and try to use the respondent’s own words.

Recording the Interview
*

Survey Research: Interviews and Telephone Surveys
6

Advantages of Telephone Surveys
Disadvantages of Telephone Surveys
No field staff needed
Simpler monitoring/prevention
of interviewer bias
Difficulty in obtaining in-depth responses
Many numbers are unlisted
Quick and inexpensive
The poor and transient may not own phones
Generally yield a low nonresponse rate
Easy and inexpensive follow-up
If the survey is global, the cost could be prohibitive
Possible high refusal rates
*

Survey Research: Interviews and Telephone Surveys
6
Sampling procedure employed in telephone surveys in which random numbers are used to obtain unlisted numbers
Random Digit Dialing
*

Survey Research: Interviews and Telephone Surveys
6
Branching Procedure
Interview technique used to narrow down sensitive responses, such as income, into less threatening categories or ranges
Techniques Employed In Telephone Surveys
*

Survey Research: Interviews and Telephone Surveys
6
Dark Figure of Crime
Crime that is unmeasured by official statistics or that has not come to the notice of police
Victim Surveys
Surveys in which subjects are asked to report alleged victimizations

Victim Surveys in Criminal Justice
*

*

Survey Research: Interviews and Telephone Surveys
6
NCVS National Crime Victims Survey

Sampling
Panel Design

Survey Research: Interviews and Telephone Surveys
6
Cost of large samples

False reporting

Mistaken interpretation
of incidents
Sampling bias
Overreporting and/or underreporting

Some Problems in Victim Surveys
Memory failure, decay, and telescoping
Interviewer effects
Coding and
mechanical error

Problems measuring certain crimes
*

Matching up Sampling, Fixing Survey Issues, and Research
CJ3675 Unit 5

In Unit 5, we will be exploring survey design, matching types of survey to research, and discussing different survey
methodology. Surveys provide great information and can be use as a data-gathering resource. Depending on the
survey type, many surveys are one-time data-collection methods, unless it is part of a longitudinal study.
Additionally, surveys are more often than not fairly short. Surveys can be used to gather information on a
participant’s perception of different issues such as the police. Furthermore, self-report surveys ask participants to
report victimization, offending, or both, as well as deviant behaviors.

Selecting a Survey Method

Surveys can be very useful tools in terms of data collection. Right now, there are a few different types of surveys.
First, there is the traditional in-person survey questionnaire, telephone surveys, online surveys, as well as mail
surveys. Each survey type has different pros and cons that will be covered in your reading and resources for this
week. When designing a survey, it is important to first ask a few questions, who is my target population? What type
of technology will they have access to? What age range is the intended survey population? Is the survey local or do
we need a nationally representative sample?

As discussed last week, we should check out what the consortium has available to see if a preexisting database can
be utilized for the intended research. If not, then we can move on to designing a survey. At first glance, survey
design appears to be cut and dry. Unfortunately, this is not true. A lot of work and effort is put into survey design and
researchers who have been designing surveys for decades still run into instrumentation issues. Prior to submitting
the required Institutional Review Board (IRB) paperwork for permission to engage in research we need to figure out
what methods will be used, and we will need to include a copy of the survey for review by the board.

What type of setting are we looking to collect data at or in? Are we trying to get a local representative and maybe we
want to go sit somewhere in the local mall or a popular local establishment. If we are targeting younger generations
that are married to their technology we may want to consider an Internet survey. Do we need to have a sample the
represents the continental United States of America and have a lot of time available but financial resources are
limited? We may want to consider doing a telephone survey. Do we have a lot of free time, a good financial resource
and need a nationally representative sample? We may want to consider a mail survey. Depending on the population,
the time line, the amount of free time or assistance available from volunteers, and the amount of financial resources
dictates the type of survey to be used.

Survey Design

Once we have settled on what type of survey data collection, we need to decide what we would like to measure,
what type of questions are needed for those measurement, and the overall design of the survey. Over the course of
this class we have discussed two main types of variables. The independent variable, or predictor variable, and the
dependent variable, the outcome variable; we run different analyses to see if any relationships exists between the
variables and how the independent variable impacts the dependent variable. In previous units we also discussed
operationalization. Operationalization takes a concept and changes it into a variable by answering how is it being
measured. In terms of survey design we can, for example, think of theoretical concepts and try to find questions to
measure them. If we wanted to test an opportunity model on cybervictimization, we would need to start by defining
what we mean theoretically.

Theoretically speaking, an opportunity model is the combination of two popular theories; Routine Activities Theory
and Lifestyles Theory. Under Routine Activities Theory, we know that lack of capable guardian, presence of a
motivated offender, and a suitable target that exist spatio-temporally increase the chance of predatory crime. This
model speaks to how we get to lack of guardianship, and highlights that this component is the most important, and
that motivated offenders are always present, however, it does not explore target suitability. Lifestyles Theory
addresses that issue and states that people who engage in certain types of behaviors have an increased chance of
victimization. Collectively, the two theories (Routine Activities Theory and Lifestyles Theory) are known as L-RAT or
Opportunity Theory. At its conception, the theory only applied to traditional in-real-life environments but the theory
has since been tested and found applicable to the digital environment as well.

In designing this survey, we would need to operationalize the different components of the theory. A popular
approach is to review previous literature and find out what questions have been used before to measure these
concepts. For instance, in reviewing the literature, we will find that opportunity factors are broken down into
frequency, risk, and self-guardianship. Since we are talking about a digital environment frequency would refer to or
be operationalized by questions that asks how much time a participant spends on the Internet, the number of social
media accounts they have, how much time they spend browsing and posting on social media, and perhaps the
amount of time they spend on their cellular phone. Risk could be operationalized in a digital environment as the type
of websites participants’ use, how willing they are to engage in a conversation with someone they have never met in
real life, and how much personal information they are willing to give out over the Internet. Self-guardianship can be
defined as privacy settings or willingness to block individuals who harass or threaten them, and could be
operationalized by asking the participants what type of privacy settings they use, if their profile is public or private,
and if they block individuals who threaten or harass them.

Once we have established our concepts and have operationalized them, we can start to determine which questions
can be dropped because another thing we want to be mindful of with survey research and using survey
questionnaires is how long it will take a participant to complete. If we are surveying college students we want to be
mindful that we are asking professors to donate class time and surveys should be limited to ten to fifteen minutes.
We also need to be mindful of the participants, if a potential participant hears it will take thirty minutes or opens up a
thirty page survey, they may be less inclined to participant, which is well within their ethical rights. Remember, all
participation is voluntary, responses need to be anonymous and if they cannot be kept anonymous they must be
confidential, and a participant can skip any question or stop participating at any time. Once the design is set, it is
time to start reviewing and drilling down. If you remember back to the previous discussion, one of the threats to
internal validity is instrumentation. We cannot change the questionnaire once it has been given out because it will
impact our research and study. A way to mitigate this threat is to use focus groups of similar participants from a
different pool that will not be surveyed and to handle any survey issues.

Survey Issues

In constructing a survey it is important that we are mindful of the accompanying issues. The most popular issues are
using bias/leading questions, double-barreled questions, providing preset answers that are not mutually exclusive
(overlapping answers) or exhaustive (do not include an answer for everyone). There are ways to identify these
issues and fix them. The first way is using a focus group. A focus group is a great way to pre-test survey
questionnaires on a group that is similar but not from the same sampling pool. We do not want to use a focus group
from our sampling pool because it would cause other threats to internal validity such as test/re-test. We want to use
a population that is similar. For instance, if we wanted to survey Kean University students about their National
Football League fandom, we could gather focus group participants from Jersey City State University because both
groups are college students but we are not biasing our selection pool.

Let’s discuss survey issues using the NFL fandom example. It is fascinating to meet different people and figure out
where their fandom lies and how it came to be. For instance, sometimes you will find a person who was born and
raised in New Jersey and their entire family is from New Jersey and are a fan of the Miami Dolphins football team.
Did this happen because they were born during the time leading up into the Dan Marino Era? Do they really like the
colors orange and green? Do they have a peer group who cheers for the Dolphins? In order to find out we want to
engage in survey research to find out. One of the questions we will need to ask is what NFL team they cheer for or
consider “their” team. One of the issues we have discussed is the categories in regards to the answers are not
exhaustive, meaning not all of the options are listed. For this question, at first look we may believe we only need
thirty-two answers because currently, there are thirty-two teams in the NFL. However, we would cause an issue to
occur. Did we provide an option for someone who is not a NFL football fan? How about the anomalies that exist who
root for multiple teams (which we could then debate if they are a true fan)? We did not. We would need to include at
minimum thirty-four responses.

Now let us arguably say for purposes of this example that we included multiple teams in the answer sets. We will
then have caused a different issue. We would have created an issue because the categories will not be mutually
exclusive meaning that participants would be able to select more than one answer (unless the question is built for
this type of answer, this practice should be avoided). This creates an issue because then the participant would fall
into two categories and their answers would need to be excluded or the participant may not answer the question
because of confusion. It is best to allow questions to only have one answer in each answer selection.

Additionally, focus groups help to identify and avoid bias/leading questions, questions that ask participants about
things that would not be in their average knowledge base, as well as double-barreled questions. Double-barreled
question and answers provide multiple question or multiple answers that may be unrelated. For instance, using our
NFL example, if we asked are you a football fan and did you go to the game on Sunday? We are asking participants
two questions in one question they may be a fan but did not go to the game. In order to fix this issue we would break
this down into two separate questions. Furthermore, if the question asks if they purchased food from a vendor while
attending the game, and we wrote answer a) yes, beer and hotdogs, it is plausible that someone purchased a hot
dog independent of a beer and vice versa. We would make sure that the answers were separated.

(CSLO 5, CSLO 6, CSLO 7))

References

Cohen, L. & Felson, M. (1979). Social change and crime rate trends: A routine activity approach. American
Sociological Review. 44(4). pp. 588-608.

Hayton, A. C. (2017). Understanding factors that impact cyberbullying offending and victimization. (Unpublished
doctoral dissertation). University of New Haven, West Haven, C.T.

Hindelang, M., Gottfredson, M., & Garofalo, J. (1978). Victims of personal crime: An empirical foundation for a
theory of personal victimization. Ballinger Publishing Company: Cambridge, Mass.