Skip Navigation


American Journal of Epidemiology Advance Access originally published online on March 22, 2007
American Journal of Epidemiology 2007 165(11):1336-1342; doi:10.1093/aje/kwm019
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow Figure
Right arrow All Versions of this Article:
165/11/1336    most recent
kwm019v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (7)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Edwards, S. L.
Right arrow Articles by Tom-Orme, L.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Edwards, S. L.
Right arrow Articles by Tom-Orme, L.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

American Journal of Epidemiology Copyright © 2007 by the Johns Hopkins Bloomberg School of Public Health All rights reserved; printed in U.S.A.

PRACTICE OF EPIDEMIOLOGY

Development and Use of Touch-Screen Audio Computer-assisted Self-Interviewing in a Study of American Indians

Sandra L. Edwards1, Martha L. Slattery1, Maureen A. Murtaugh1, Roger L. Edwards1, James Bryner2, Mindy Pearson2, Amy Rogers1, Alison M. Edwards1 and Lillian Tom-Orme1

1 Division of Clinical Epidemiology, Department of Internal Medicine, School of Medicine, University of Utah, Salt Lake City, UT
2 Health Research Center, Department of Family and Preventive Medicine, School of Medicine, University of Utah, Salt Lake City, UT

Reprint requests to Sandra L. Edwards, Department of Internal Medicine, University of Utah School of Medicine, AC 230, 30 North 1900 East, Salt Lake City, UT 84132 (sandie.edwards{at}hsc.utah.edu).

Received for publication June 28, 2006. Accepted for publication November 8, 2006.


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 References
 
This article describes the development and usability of an audio computer-assisted self-interviewing (ACASI) questionnaire created to collect dietary, physical activity, medical history, and other lifestyle data in a population of American Indians. Study participants were part of a cohort of American Indians living in the southwestern United States. Data were collected between March 2004 and July 2005. Information for evaluating questionnaire usability and acceptability was collected from three different sources: baseline study data, auxiliary background data, and a short questionnaire administered to a subset of study participants. For the subset of participants, 39.6% reported not having used a computer in the past year. The ACASI questionnaires were well accepted: 96.0% of the subset of participants reported finding them enjoyable to use, 97.2% reported that they were easy to use, and 82.6% preferred them for future questionnaires. A lower educational level and infrequent computer use in the past year were predictors of having usability trouble. These results indicate that the ACASI questionnaire is both an acceptable and a preferable mode of data collection in this population.

data collection; epidemiologic methods; Indians, North American; interviews; pilot projects; questionnaires; software; user-computer interface

Abbreviations: ACASI, audio computer-assisted self-interviewing; DHQ, Diet History Questionnaire; EARTH, Education and Research Towards Health; HLPA, Health, Lifestyle, and Physical Activity Questionnaire


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 References
 
Audio computer-assisted self-interviewing (ACASI) is a new method of collecting questionnaire data for epidemiologic studies (1, 2). In ACASI, the respondent completes a self-administered questionnaire using a computer, with recorded audio simultaneously reading the questions, answer categories, or instructions as they appear on the screen. ACASI questionnaires have mainly been used to ask sensitive questions, since answers about risky or sensitive behaviors given to personal interviewers are more likely to be biased (1–10).

There are other advantages of using ACASI: Trained interviewers can give instructions but do not have to be present during the entire interview or can be present at less than a 1:1 ratio; ACASI can alleviate literacy problems, since questions and answers are read aloud; and the questionnaire can be administered in multiple languages. ACASI has preprogrammed skip patterns, thus eliminating navigational problems associated with self-administered paper questionnaires. Preprogrammed range checks can be built in to identify improbable answers. Answers are entered directly into the computer, making data available almost immediately. ACASI allows questionnaire designers to use graphics, making it possible to use pictures, which are comparable to show cards.

In this paper, we report on the development, usability, and acceptability of two ACASI questionnaires created for a study of American Indians. We describe their development and present usability and acceptability data from a short usability questionnaire administered to a subset of participants. We present background data and discuss the characteristics of participants who had trouble using the questionnaires.


   MATERIALS AND METHODS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS
 DISCUSSION
 References
 
Study background
The Navajo Education and Research Towards Health (EARTH) Study is part of a multicenter study funded to 1) develop valid methods of collecting data on diet, physical activity, and lifestyle factors for use in a cohort of American Indians and Alaska Natives and 2) develop methods of providing rapid feedback to participants and local communities that can be used to monitor health and plan health programs. Participants are aged 18 years or older, self-identify as American Indian or Alaska Native, have given informed consent, understand English or Navajo, and are eligible to receive health care from the Indian Health Service. Participation involves visiting a study site to complete study questionnaires and have blood pressure, blood lipid, and body size measurements taken. The institutional review boards of the University of Utah, the Indian Health Service, and the Navajo Nation approved the study. Additionally, local health boards and chapters within local health boards approved and supported the study.

Questionnaire development
Two questionnaires were developed by study investigators—the Diet History Questionnaire (DHQ) and the Health, Lifestyle, and Physical Activity Questionnaire (HLPA)—and programmed for administration by touch-screen ACASI. Questionnaire design, computerization, and usability testing were conducted at the EARTH coordinating center and the field offices.

The initial visual design and layout of the computer screens was developed using established principles for developing self-administered paper questionnaires and computer-assisted interviewing (11–15). Each screen contained only one question, except for multipart frequency and serving size/intensity questions in the diet and physical activity sections. No drop-down boxes were used. The use of graphics and color was kept to a minimum. Usability standards were applied in the design process, with the goals being consistent screen design, visual discrimination among various elements, adherence to normal reading behavior, displaying instructions at a point appropriate to the task, and eliminating clutter and unnecessary information that could distract users from immediate tasks (12, 15–19). Each screen had NEXT, HELP, LANGUAGE, and BACK buttons. The NEXT button had to be selected to go on to the next question; the HELP button went to a screen with additional information on the question; the LANGUAGE button changed the language of the audio; and the BACK button took the participant back to the previous screen. English and Navajo scripts of the finalized questions were prepared, and audio files were recorded and saved as computer waveform audio files. Because of the length of the questionnaires, answer categories were not recorded and read. The waveform audio files were edited for uniform volume and consistent silence at the beginning and end and then converted to the MP3 format because of file-size considerations.

Usability was tested in three phases evaluating the characteristics of learnability, efficiency, memorability, error handling, and subjective satisfaction (18, 20). First, students from a graduate nutrition class completed the questionnaires while being observed and videotaped by study staff. Students were encouraged to think aloud while completing the questionnaires. These videotapes were coded using a standardized coding scheme. During phases 2 and 3, questionnaires were administered to volunteers at the field offices. Because it was not possible to videotape these volunteers, a trained observer coded behavior as the questionnaires were completed. Changes to the design of the ACASI screens were made after each phase. Table 1 lists the final screen types used in the questionnaires. Figure 1 shows an example of a DHQ single-answer categorical response screen. The Web figure (posted on the Journal's website (http://aje.oxfordjournals.org/)) contains an example of a DHQ frequency and serving-size screen question sequence.


View this table:
[in this window]
[in a new window]

  		TABLE 1. Final screen types developed for two audio computer-assisted self interviewing questionnaires, Navajo Education and Research Towards Health (EARTH) Study, southwestern United States, 2004–2005

 

Figure 1
View larger version (24K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		FIGURE 1. Example of a single-answer categorical response question from the Diet History Questionnaire, Navajo Education and Research Towards Health (EARTH) Study, 2004–2005.

 
Data collection
The ACASI questionnaires were administered on touch-screen panels. Study staff gave participants standardized instructions that included information on using headphones, answering questions, changing answers, clearing numeric answers, and using the NEXT, BACK, HELP, and LANGUAGE buttons. The staff remained with the participant through the first few questions of each questionnaire to make sure the participant did not have additional questions. Between March 15, 2004, and July 15, 2005, 2,946 participants completed the study visit.

Paradata
The ACASI software recorded background paradata (data about the process of data collection obtained through time stamps, keystroke files, and trace files) that identified usability issues (17, 21). Paradata were available for 1,612 of the 2,946 participants. Variables created from the paradata included completion times and numbers of times that the audio was repeated, the skip pop-up appeared, questions were skipped, the range check pop-ups appeared, and the HELP and LANGUAGE buttons were used.

Usability questionnaire
Participants who completed a study visit between April 14, 2005, and July 15, 2005 (n = 688) were asked to complete a self-administered 14-question paper usability questionnaire at the end of the visit. Of these persons, 604 (87.8 percent) completed the usability questionnaire. Participants were asked about their computer use during the past year; whether they agreed or disagreed (on a five-item Likert scale) that 1) more instructions about using the touch screens were needed, 2) the touch screens were very easy to use, and 3) they enjoyed using the computer to complete the questionnaire; which questionnaire method offered the most privacy; which questionnaire method they would prefer in the future; how often they had used the audio; and seven "yes/no/not applicable" questions about specific problems they might have encountered while completing the questionnaires.

Statistical analysis
Demographic data, including data on education, employment, age, and marital status, were collected in an interviewer-administered questionnaire. Information on perceived health status, income, language spoken at home, and identification with Native culture was collected in the HLPA. Data from the usability questionnaire and paradata were linked to the data from the study visit.

Responses to usability questionnaire questions with a five-item Likert scale were collapsed to a three-item scale: agree (strongly agree/agree), disagree (strongly disagree/disagree), and neutral (neither agree nor disagree). Computer use in the past year was categorized as none, infrequent (<12 times per year or 1–3 times per month), or frequent (once per week or more). Participants who said they 1) had a hard time understanding how to use the touch screens, 2) had trouble changing answers, or 3) had trouble with warning boxes were considered to have had difficulty using ACASI, and others were not. Participants were classified as having usability trouble if they 1) agreed that more instructions were needed, 2) disagreed that the questionnaire was easy to use, 3) said that they would prefer an in-person or self-administered paper questionnaire in the future, or 4) were classified as having difficulty using ACASI; others were considered not to have had usability trouble.

Statistical analysis was completed using the SAS system for Windows, version 9.0 (SAS Institute, Inc., Cary, North Carolina). Demographic characteristics for the study, usability questionnaire, and paradata participants are presented. Differences in the distribution of a categorical variable between two groups were assessed using chi-squared tests. Stepwise multiple logistic regression was used to identify the best predictors for having trouble completing the ACASI questionnaire.


   RESULTS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
DISCUSSION
 References
 
Study participants, those with paradata, and those who completed the usability questionnaire were similar demographically (table 2). More study participants than usability participants (45.7 percent vs. 38.1 percent) reported education beyond high school.


View this table:
[in this window]
[in a new window]

  		TABLE 2. Characteristics of study participants, usability questionnaire participants, and participants with paradata in the Navajo Education and Research Towards Health (EARTH) Study, southwestern United States, 2004–2005

 
Usability questionnaire participants had high acceptance of the ACASI questionnaires, with 96.0 percent reporting finding them enjoyable to use, 97.2 percent reporting that they were easy to use, and 82.6 percent preferring them for future questionnaires (table 3). However, 62 percent indicated that they desired more directions, and 10.6 percent reported having difficulty using ACASI.


View this table:
[in this window]
[in a new window]

  		TABLE 3. Distribution of responses to questions on the usability of ACASI* among usability questionnaire participants (n = 604) in the Navajo Education and Research Towards Health (EARTH) Study, southwestern United States, 2005

 
The mean completion time for the 1,612 paradata participants was 36 minutes for the DHQ and 29 minutes for the HLPA (data not shown). The DHQ was completed in less than 30 minutes by 34.7 percent of participants; 60.5 percent took 30–59 minutes, and 4.8 percent took 1 hour or more. The HLPA was completed in less than 30 minutes by 56.4 percent of participants; 43.1 percent took 30–59 minutes, and 0.5 percent took 1 hour or more. The HELP button was used by 7.8 percent of participants with the DHQ and 40.1 percent with the HLPA. The question text audio was replayed by 46.7 percent of participants with the DHQ and 47.9 percent with the HLPA.

The paradata revealed a usability problem. When an answer was selected, the button darkened. When the NEXT button was selected, it did not darken, resulting in its being selected twice in some instances. The skip/return pop-up then appeared, requiring the participant to skip or return to the question. Thirty-seven paradata participants did not receive this pop-up, 90 received it once, and 1,485 received it more than once, resulting in 305 participants having one or more questions skipped.

Responses to the usability questions varied for some demographic characteristics, as well as by computer experience in the past year, DHQ and HLPA completion time, and number of skip pop-ups received (table 4). Of the factors evaluated, lower educational level and less frequent computer use in the past year were associated with usability trouble (data not shown).


View this table:
[in this window]
[in a new window]

  		TABLE 4. Distribution of responses to questions on the usability of ACASI{dagger} among usability questionnaire participants (n = 604) in the Navajo Education and Research Towards Health (EARTH) Study, by participant characteristic, southwestern United States, 2005

 

   DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
References
 
The usability and acceptability of the ACASI questionnaire was comparable to that of other studies (2, 3, 22, 23). Acceptance was high, even though some participants experienced problems using the questionnaires and many (39.6 percent) had not used a computer during the past year. However, we identified a few problems with ACASI usability and acceptability. A considerable proportion of older participants reported more difficulty using ACASI (25.7 percent of those aged 60–69 years and 40.0 percent of those over age 69 years), and a majority wanted more instructions (70.3 percent of those aged 60–69 years and 90.0 percent of those over age 69 years). Acceptability was lower for participants with less education, who were least likely to prefer ACASI for the future, and the youngest participants, who found it the least private. In addition, only 71.4 percent of participants used the audio all or most of the time, with lower audio use being observed among participants who were retired (60.0 percent), spoke mostly Navajo at home (63.8 percent), or reported excellent or poor health (59.3 percent and 55.6 percent, respectively).

ACASI questionnaires were well accepted and usable in this mostly young (75.0 percent were aged 18–49 years) population and allowed for feedback to be given immediately after the study visit was completed as opposed to being mailed. Using ACASI allowed for less than a 1:1 ratio of staff to participants, thus reducing data collection costs. Investigators should consider using ACASI as an alternative to in-person or self-administered questionnaires in studies where participants come to a central study-visit site or for questions on sensitive topics. These methods also may be useful for Web-based data collection.


   ACKNOWLEDGMENTS
 
This study was funded by grant CA 088958 from the National Cancer Institute.

The authors acknowledge the contributions of Joan Benson, Clarina Clark, Kelly Cunningham, Carmen George, and the staff at the Navajo Nation field centers.

The contents of this manuscript are solely the responsibility of the authors and do not necessarily represent the official view of the National Cancer Institute.

Conflict of interest: none declared.


   References
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 References
 

  1. Aday LA. Designing and conducting health surveys: a comprehensive guide (1996) San Francisco, CA: Jossey-Bass, Inc.
  2. Metzger DS, Koblin B, Turner C, et al. Randomized controlled trial of audio computer-assisted self-interviewing: utility and acceptability in longitudinal studies. HIVNET Vaccine Preparedness Study Protocol Team. Am J Epidemiol (2000) 152:99–106.[Abstract/Free Full Text]
  3. Katz LM, Cumming PD, Wallace EL, et al. Audiovisual touch-screen computer-assisted self-interviewing for donor health histories: results from two years experience with the system. Transfusion (2005) 45:171–80.[CrossRef][Web of Science][Medline]
  4. Lessler JT, O'Reilly JM. Mode of interview and reporting of sensitive issues: design and implementation of audio computer-assisted self-interviewing. NIDA Res Monogr (1997) 167:366–82.[Medline]
  5. Thornberry J, Bhaskar B, Krulewitch CJ, et al. Audio computerized self-report interview use in prenatal clinics. Comput Inform Nurs (2002) 20:46–52.[CrossRef][Medline]
  6. Des Jarlais DC, Paone D, Milliken J, et al. Audio-computer interviewing to measure risk behaviour for HIV among injecting drug users: a quasi-randomised trial. Lancet (1999) 353:1657–61.[CrossRef][Web of Science][Medline]
  7. Ellen JM, Gurvey JE, Pasch L, et al. A randomized comparison of A-CASI and phone interviews to assess STD/HIV-related risk behaviors in teens. J Adolesc Health (2002) 31:26–30.[CrossRef][Web of Science][Medline]
  8. Boekeloo B, Schiavo L. Self-reports of HIV risk factors by patients at a sexually transmitted disease clinic: audio vs written questionnaires. Am J Public Health (1994) 84:754–60.[Abstract/Free Full Text]
  9. Turner C, Ku L, Rogers SM, et al. Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science (1998) 280:867–73.[Abstract/Free Full Text]
  10. O'Reilly JM, Hubbard ML, Lessler JT, et al. Audio and video computer assisted self-interviewing: preliminary tests of new technologies for data collection. J Off Stat (1994) 10:197–214.
  11. Christian LM, Dillman DA. The influence of graphical and symbolic language manipulations on responses to self-administered questions. Public Opin Q (2004) 68:57–80.[Abstract/Free Full Text]
  12. Couper MP. Designing effective web surveys. Workshop presentation at the conference of the American Association for Public Opinion Research, St. Petersburg Beach, Florida, May 19, 2002. (http://www.amstat.org/meetings/jsm/2003/onlineprogram/index.cfm?fuseaction=activity_details&sessionid=201155).
  13. DeRouvray C, Couper MP. Designing a strategy for reducing "no opinion" responses in web-based surveys. Soc Sci Comput Rev (2002) 20:3–9.[Abstract/Free Full Text]
  14. Dillman DA. Mail and Internet surveys: the tailored design method (2000) New York, NY: John Wiley and Sons, Inc.
  15. Bosley J, Conrad FG, Uglow D. Pen CASIC: design and usability. In: Computer assisted survey information collection—Couper MP, Baker RP, Bethlehem J, et al, eds. (1998) New York, NY: John Wiley and Sons, Inc. 521–41.
  16. Couper MP. The application of cognitive science to computer assisted interviewing. In: Cognition and survey research—Sirken MG, Herrmann DJ, Schechter S, et al, eds. (1999) New York, NY: John Wiley and Sons, Inc. 277–300.
  17. Couper MP. Usability evaluation of computer assisted survey instruments. Soc Sci Comput Rev (2000) 18:384–96.[Abstract/Free Full Text]
  18. Schneider SJ, Edwards B. Developing usability guidelines for AudioCasi respondents with limited literacy skills. J Off Stat (2000) 16:255–71.
  19. Hansen SE, Couper MP. Usability testing to evaluate computer-assisted instruments. In: Methods for testing and evaluating survey questionnaires—Presser S, Rothgeb JM, Couper MP, et al, eds. (2004) New York, NY: John Wiley and Sons, Inc. 337–60.
  20. Rubin J. Handbook of usability testing: how to plan, design, and conduct effective tests (1994) New York, NY: John Wiley and Sons, Inc.
  21. Casper R, Couper M. Using keystroke files to assess respondent difficulties with an AudioCasi instrument. In: In: Proceedings of the Section on Survey Methods, American Statistical Association (1997) Alexandra, VA: American Statistical Association. 239–44.
  22. Jones R. Survey data collection using audio computer assisted self-interview. West J Nurs Res (2003) 25:349–58.[Abstract/Free Full Text]
  23. Perlis TE, Des Jarlais DC, Freidman SR, et al. Audio-computerized self-interviewing versus face-to-face interviewing for research data collection at drug abuse treatment programs. Addiction (2004) 99:885–96.[CrossRef][Web of Science][Medline]

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?


This article has been cited by other articles:


Home page
 AJPHHome page
J. Gupta, D. Acevedo-Garcia, D. Hemenway, M. R. Decker, A. Raj, and J. G. Silverman
Premigration Exposure to Political Violence and Perpetration of Intimate Partner Violence Among Immigrant Men in Boston
Am J Public Health, March 1, 2009; 99(3): 462 - 469.
[Abstract] [Full Text] [PDF]

Home page
 Br J Soc WorkHome page
A. Morgan and S. Fraser
Looked After Young People and their Social Work Managers: A Study of Contrasting Experiences of Using Computer-Assisted-Self-Interviewing (A-CASI)
Br. J. Soc. Work, February 12, 2009; (2009) bcp009v1.
[Abstract] [Full Text] [PDF]

Home page
 Inj. Prev.Home page
D G Redwood, K D Hagan, R D Perkins, H B Stafford, L J Orell, and A P Lanier
Safety behaviours among Alaskan Native and American Indian people living in Alaska
Inj. Prev., February 1, 2009; 15(1): 30 - 35.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow Figure
Right arrow All Versions of this Article:
165/11/1336    most recent
kwm019v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (7)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Edwards, S. L.
Right arrow Articles by Tom-Orme, L.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Edwards, S. L.
Right arrow Articles by Tom-Orme, L.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?