Quality control issues are very important to the Kansas Behavioral Risk
Factor Surveillance (BRFSS) program. The data we collect must be as accurate
as possible to ensure that it represents the self-reported health and beliefs
of the people of Kansas. To provide consistent and timely quality control,
a number of tasks are performed before and during the data collection process,
and after data collection is complete.
First, prior to data collection, interviewers are extensively trained.
Newly hired interviewers participate in a two-day, eight-hour online training
developed by the Centers for Disease Control and Prevention (CDC) and
located on CDC’s website at www.cdc.gov/brfss/training.htm. This
BRFSS Interviewer Training is interactive and includes computer testing
after each section is completed.
Prior to implementation of each new survey, the Survey Supervisor trains
the interviewers on the actual content of the survey. Each survey is reviewed
question by question to ensure that there is understanding of the questions
being asked, the possible responses to each question, and proper pronunciation.
This is followed by a debriefing session to address any questions or concerns
that the interviewers may have before they begin interviewing.
Each new interviewer is then assigned to an experienced interviewer for
further training. The trainee interviewers sit with the experienced interviewers
as they make phone calls so they can experience what a night is like as
an interviewer and observe the complications that may arise while they
are interviewing on their own. This is a peer training and allows to the
trainees to see and hear “live” interviews and ask specific
questions that may be unique to each individual interview.
Additionally, each survey has a test study where the trainees can practice
the survey without actually being on the telephone. Interviewers usually
practice on a test study for several days before they are ready to call
respondents and conduct a “real” interview. This allows them
to see the questions on the computer monitor and to get acquainted with
the different skip patterns and response categories that are often unique
to each individual survey. During this time, new interviewers are encouraged
to use each other as mock respondents so they can become familiar with
using the telephone and the computer at the same time. Finally, the trainee
interviewers meet with their trainer and they discuss any questions or
concerns they may have before going “live”. Once the trainer
and the trainee interviewer feel the trainee is ready, he/she is assigned
a station and begins calling on the actual study.
Second, the Survey Supervisor performs interviewer monitoring monthly.
The monitoring consists of the Survey Supervisor listening in on each
interviewer as they perform a survey. The Survey Supervisor completes
a monitoring form that is based on the BRFSS User’s Guide recommendations.
This form is then shared with the interviewer as a performance review.
This gives the interviewer immediate feedback regarding attitude, interviewing
techniques and compliance with BRFSS protocol. Any concerns, questions,
or retraining issues are discussed with the interviewer. After the form
is discussed, both the Survey Supervisor and the Interviewer sign the
form, showing that it has been reviewed and any issues have been discussed.
On the average, each Interviewer is monitored at least once per month.
New interviewers are monitored more frequently to ensure that they are
following CDC’s BRFSS protocol. The monitoring forms are kept confidential
and are tracked on a monthly spreadsheet.
Third, the data is collected in-house rather than contracted out to a
research firm. This allows Kansas BRFSS staff to perform frequent reviews
of the data to ensure that it is being collected and stored correctly.
If errors are detected any time during the month the data can be reviewed
and the corrections made quickly without major interruptions to the data
collection process or extensive corruption of the data. In-house data
collection allows for more reactive and timely response to protocol infractions
and data errors.
Fourth, the data is edited on a monthly basis. The CDC has developed
a data-editing program used specifically for the BRFSS. Each month the
data is forwarded to the CDC for review. This allows us to identify and
correct any data errors and/or outliers in a timely manner.
The processes we have implemented and the CDC’s BRFSS protocol
are important to ensure quality control. Training, monitoring, in-house
data collection and data editing allows for us to ensure that the sample
we collect is of the best quality possible. This is an on-going task,
which helps to ensure that the BRFSS data are highly representative of
the health behaviors and opinions of the Kansas people.