Communication Institute for Online Scholarship
Communication Institute for Online
Scholarship Continous online service and innovation
since 1986
Site index
 
ComAbstracts Visual Communication Concept Explorer Tables of Contents Electronic Journal of Communication ComVista

Article from ejc/rec EJCREC, Vol. 11, #3 and 4, 2001
EJC logo
The Electronic Journal of Communication / La Revue Electronique de Communication

Volume 11 Numbers 3 and 4, 2001

PARTICIPATORY DESIGN FOR BETTER INTERACTIVE HEALTH COMMUNICATION:
A STATEWIDE MODEL IN THE USA*

Linda Neuhauser
University of California, Berkeley

Abstract. Interactive health communication (IHC) has great potential to overcome the weaknesses of traditional health education and improve people's health worldwide. Currently there are many barriers that prevent large numbers of people from effectively using online health information. The vast majority of health sites are not accessible to many people with disabilities, people with low literacy, and people using a non-dominant language.

Participatory design techniques, previously advanced in many fields internationally, are powerful tools to develop usable, effective Web sites. Participatory design involves the users as collaborators in the development of accessible interfaces and relevant content. Usability processes and remote collaboration software are now making it possible for people around the world to jointly create and test Web sites. The World Wide Web Consortium has defined practical international guidelines to ensure that Web sites are accessible for people with disabilities.

This paper provides an example of how participatory design was used by hundreds of parents and people with disabilities to create a health Web site intended to benefit 33 million residents of the state of California, U.S.A. The basic approaches used in this model could be adapted to develop improved IHC in other countries.

Key words: participation, user-centered design, usability design, interactive health communication, health informatics, online health information, and disability.

As we begin this century, interactive health communication (IHC) is emerging as a field with great potential to improve people's health and well being worldwide. IHC has been defined as

the interaction of an individual--consumer, patient, caregiver or professional--with or through an electronic device or communication technology, to access or transmit health information, or to receive or provide guidance and support on a health-related issue. (Robinson, Patrick, Eng & Gustafson, 1998)

IHC applications refer to software programs or modules that interface with users, rather than the hardware and infrastructure technologies that run these applications (Science Panel on Interactive Communication and Health (SciPICH), 1999). This paper focuses primarily on IHC applications that involve computers to communicate health information to consumers. Such IHC applications serve many purposes: to inform, influence and motivate consumers (National Cancer Institute, 1989). They relay information, enable informed decision-making between patients and providers, promote healthy behaviors, peer information exchange, self-care, and help manage demand for health services (SciPICH, 1999; Agency for Health Care Policy and Research, 1997; Ferguson, 1996).

An estimated 64 million people in the U.S. searched for health information on the Internet in 2000 (Horrigan, 2002). Therefore, there is an unprecedented opportunity for people to use information technologies to link to the knowledge they need to make good choices about health care and daily living.

The Challenge to do Better with Health Communication

In the U.S., as in most industrialized countries, the vast majority of health resources are used to manage clinical diseases. However, it is estimated that half of all deaths are attributed to behavioral and social factors such as diet, smoking, alcohol use, and exercise (McGinnis & Foege, 1993).  Currently, less than 5% of the U.S. health care budget is allocated to reduce preventable risks due to these factors. (Centers for Disease Control and Prevention, 1992; Health Care Financing Administration, 2000). A recent Institute of Medicine report concluded that "Behavioral and social interventions therefore offer great  promise to reduce disease morbidity and mortality, but as yet their potential to improve the public's health has been relatively poorly tapped" (Smedley & Syme, 2000).

During the past century, the main approach to intervening on behavioral factors has been through health education. Unfortunately, these interventions, especially those based on educational materials, have often shown disappointing results. (Curro, Lanni, Scipione, Grimaldi & Mastoriacovo, 1997; Powell, Tanz, Uyeda, Gaffney & Sheehan, 2000; Walpole, Watson, Moore, Goldblatt & Bower, 1997; Witte, Cameron, Lapinski & Nzyuko, 1998; Assaf, Cummings, Graham, Mettlin & Marshall, 1985; Heller, Elliott, Bray & Alabaster, 1989; Kirby, Harvey, Claussenius & Novar, 1989; Silvestri & Flay, 1989; Stergachis, Newmann, Williams & Schnell, 1990). Although the messages being communicated have a solid scientific basis, and although they sometimes improve knowledge, there is little resulting change in behavior.

Why are we not more successful? Traditional health information has many limitations:

1.  It is "top down." Experts design the information, not the users.

2.  It is "non-interactive." Typically, professionals give proscriptive messages to their clients or patients. This approach can foster "disempowerment" and even negative outcomes (Smedley & Syme, 2000).

3.  It uses messages which are narrowly focused on desired outcomes, not on ways people can integrate changes into their lives.

4.  It does not usually link people to others for support.

5.  It is usually not customized: the same messages go to large groups.

6.  It is not easily locatable: the sender controls its dissemination.

7.  It is often not available to many vulnerable groups: people who do not speak the dominant language, have low literacy skills, or have disabilities.

In short, traditional health information does not usually effectively engage the users in finding what they personally need, when they need it, and in a context that empowers them to make successful change.

On the other hand, there have been notable successes (Lowe, Balanda, & Clare, 1998; Neuhauser, Schwab, Syme, Bieber & Obarski, 1998; Nicklas et. al., 1997; Puska, et al, 1981). Many of these have focused on involving the beneficiaries in the design and implementation of the educational intervention.

The Promise and Challenges of Interactive Health Communication

Theoretically, IHC applications could overcome many of the problems linked with "old media" health communication. Expected benefits include customized information, multi-media capabilities, privacy for sensitive topics, information on demand, opportunity to interact with professionals and support groups, widespread dissemination including to traditionally underserved groups, and frequent updating of information (U.S. Department of Health and Human Services, Nov 2000). Overall, IHC applications are expected to have much greater  influence on the psychosocial factors of empowerment, self-efficacy and motivation than has traditional health communication (SciPICH, 1999; Strecher et al., 1994).

Are we doing better with current IHC approaches?  As yet, few applications have been well-evaluated (Robinson et al., 1998). One barrier has been the lack of explicit criteria to evaluate health Web sites. Currently, many groups are involved in developing standardized methods to report evaluations (Kim, Eng, Deering & Maxfield, 1999; Kreps (2002); SciPICH, 1999; National Academy Press, 2000; Robinson et al., 1998; Eng, Gustafson, Henderson, Jimison & Patrick, 1999). 

Despite the paucity of evaluation, a number of studies during the past decade have shown positive results for IHC efforts. Programs using shared decision-making have shown improved health outcomes and reduced use of medical interventions (Barry, Fowler, Mulley, Henderson & Wennberg, 1995; Morgan et al., 1997). Positive outcomes were shown for women with breast cancer and AIDS patients who used computers to access support groups and information about medical decisions. (Gustafson et al., 1993; Gustafson et al.; 1994; McTavish et al., 1994). New media approaches were also shown to improve outcomes for patients with chronic disease (Greenfield & Kaplan, 1985).

Even so, there are many challenges to developing successful IHC applications. A key problem is accessibility. Analyses of Web sites in the U.S. found that only 2% used a language other than English, less than 1% were usable by people with low literacy skills, and only 6% contained referrals to specific local services. It is estimated that these barriers limit effective use by 50 million Americans (Children's Partnership, 2000). Further, lack of access is estimated to affect an estimated 30 million people with disabilities in the U.S. and 500 million worldwide who now use computers (Tobias, 2001). Obviously, many Web sites are failing to meet the needs of underserved populations, precisely the groups most in need of health communication.

Strategies to Develop Better Health Communication

The shortcomings of traditional health education suggest that engagement and collaboration of users are fundamental to develop effective IHC. Participatory design approaches can help us realize this goal.

Participatory Design

Community participation. The concept of participatory design is not new. Participatory design methods and "movements'' exist in nearly every field.  In health, the World Health Organization (WHO) has long promoted "community participation" in the design of health education to enable program staff to see potential solutions through the eyes of those most directly concerned (WHO, 1954). The principle of "community involvement" was incorporated into WHO's "Health for All strategy (WHO, 1981) and endorsed as an important element of health promotion in Canada (Labonté, 1986) and in the U.S. (U.S. Department of  Health and Human Services, 1990).  The impetus for promoting the involvement of the beneficiaries in the design of health interventions has been motivated by the growing body of research showing that participation improves health programs.

Not only is it useful to involve people in designing health interventions because it improves the relevance of the information they receive, processes of participation, in themselves, may promote the health of participants. (Syme, 1990). Participation increases personal control and self-efficacy and this has direct and positive physiological effects (Baum, 2000; McEwan, 1998)  In fact, perceived sense of control has emerged as one of the variables most predictive of health status (Marmot, 1994). As this theory evolves with more specific scientific documentation, it is likely to provide a powerful understanding of key determinants of health, and guidance to promote participation in health interventions.

Although participation is officially promoted in the health sector, it is generally overlooked in practice. "Participation" is a relative term and can be measured along a continuum of involvement (Arnstein, 1969; Ugalde, 1985). Often, what passes for participation is a weak connection between the developers and beneficiaries, rather than true collaboration. In part, this problem points to the lack of well-developed methods to engage beneficiaries (or "users") in design processes.  

Participation in user-centered design. Major contributions to specific techniques of participatory design originated in the fields of architecture, urban planning and engineering. The Scandinavian Collective Resource Approach (CRA) developed in Norway, Sweden, and Denmark during the 1960's and 70's was an important milestone. CRA was rooted in democratic values and recognized the importance of participation for multiple perspectives in the design process (Bjerknes & Bratteteig; 1995; Ehn, 1992). It acknowledged that unintended adverse consequences arise when major affected groups are not included in the design process. CRA stated that "technology is not neutral" and promoted collective understanding for the achievement of social objectives in design. The first CRA efforts involved worker empowerment and, over time, led to participatory design of computer systems.

Participatory design has been defined as "an approach to development of technological and organizational systems that places a premium on the active involvement of potential or current users in design and decision-making processes" (Computer Professionals for Social Responsibility, 2000). Unlike traditional design, participatory design is considered a "collaborative approach to design, not a rigid set of design methods" (Good, 1992).

Since the 1990's, participatory processes have rapidly proliferated in the design of computing systems and informatics (Schuler & Namioka, 1993; Greenbaum & Kyng, 1991; Muller & Kuhn, 1993). Participatory design in health informatics has now resulted in a number of interesting models, internationally. Examples include a Nigerian effort with participatory design of health informatics (Korpela et. al., 1996). In the U.S., users designed a wellness monitor for daily health information (McCormack & Forlizzi, 2000). In Canada, Departments of Counseling Psychology and Computer Science collaborated with users to  design a system to support distance art therapy (Collie, Cubranic & Booth, 1998). In Germany, patients collaborated on the design of health cards (Stark, 1998). At a U.S. hospital, users helped customize a health information system (Krabbel & Wetzel, 1998).

Reports of participatory design in IHC are still rare in the scientific literature. The work of Gustafson and his colleagues with the CHESS Project is a notable example of intense, effective user involvement (Gustafson et al., 1993; Gustafson et al., 1994). For IHC developers, a rich information source is the Computer Professionals for Social Responsibility (CPSR), an international consortium which has created a worldwide forum to promote participatory design in computer informatics. Their Web site, http://www.cpsr.org, features participatory design conferences and resources.

Related approaches to participatory design. One participatory technique to develop and test Web sites is known as usability. Usability processes are traditionally drawn from industrial design and engineering fields. Manuals about usability include rules, principles, methods and guidelines to develop effective Web sites, including content, display and interfaces (Nielsen, 2000). These guidebooks lay out specific methods to involve users in online design. The U.S. National Cancer Institute's usability site: http://www.usability.gov is one of the best resources for IHC developers, especially it's easy-to-use guidelines. Another helpful general site about usability is: http://www.useit.net.

Participatory action research is another approach which grew out of the pioneering collaborative community work of Paolo Freire (Freire, 1993). Friend notes that "A central objective of action research is to build capacity at the local lever for practitioners to influence the shape of their own work environments. Action research is an instrument for change." (Friend, 1999). Participatory action research is used as a way for users to evaluate, design, and redesign health informatics. A useful site about action research is http://www.parnet.org.

Participatory design of global online health sites may seem like an impossible goal. Fortunately, recent remote collaboration techniques are making it possible for far distant users to get involved. Remote collaboration methods have grown out of attempts to do virtual project management, which has created an array of applications known as groupware. (Johansen, 1998; Greenberg & Bohnet, 1991) A useful address to explore such collaboration is: http://www.internet.com/collaboration/.

Accessibility

Tim Berners-Lee, the inventor of the World Wide Web, stated that, "The power of the Web is in its universality. Access by everyone regardless of disability is an essential aspect." (Berners-Lee, 2000). However, the reality has been digital exclusion for many people, especially those with visual or auditory impairments. This recognition has resulted in an international effort to establish practical standards for Web accessibility and is an excellent example of participatory design. A major goal is the development of flexible, multi-modal interface techniques that can adapt to a very wide range of individuals.

A related principle is universal design or universal usability--the concept that accessible design leads to better technologies for all people, and catalyzes advances in technology. For example, having non-graphic versions of information enables Web sites to be easily accessed over cellular phones, on personal digital assistants, or through car computers. Voice recognition computing systems for the blind also have many applications for sighted people (Tobias, Burgstahler & Otten,  2000). Some universal design developers aim to create an "every-citizen interface." See http://ui4all.ics.forth.gr.call/.

An organization seeking to advance Web accessibility is the World Wide Web Consortium (W3C), an international industry consortium jointly run by the Massachusetts Institute of Technology Laboratory for Computer Science in the U.S.A. (MIT LCS), the National Institute for Research in Computer Science and Control (INRIA) in France, and Keio University in Japan. The Consortium's goal is to "lead the Web to its full potential by developing common protocols that promote its evolution and ensure its interoperability" (see http://www.w3.org). The Consortium has over 400 member organizations and is directed by Tim Berners-Lee.

W3C's Web Accessibility Initiative (WAI) is an international effort sponsored by governments and corporations interested in Web accessibility and Universal Design. Sponsors include industry, disability organizations, accessibility researchers and governments, such as the U.S.A., Canada and European countries. WAI publishes the "Web content Accessibility Guidelines." For the latest version, visit http://www.w3.org/TR/WCAG10-TECHS/. The Guidelines provide three "checkpoint" levels of accessibility:

At the simplest level ("Priority 1"), these guidelines call for Web designers to include alternative descriptions of graphics (called alt text) that can be read by screen readers--software for blind users that uses speech synthesis technologies to read text. For deaf users, Web sites that use video will need to provide written transcripts or captions.

Other key organizations that advance Web accessibility include: the University of Toronto's Adaptive Technology Resource Center http://www.utoronto.ca//atrc, and Trace Research and Development Center, University of Wisconsin-Madison: http://www.trace.wisc.edu. The Center for Accessible Special Technology (CAST) provides a practical service called "Bobby" through which designers can have their site reviewed for accessibility: http://www.cast.org/bobby/. Sites which pass approval may use the Bobby-approved icon.

In the U.S., new federal guidelines under Section 508 of the Rehabilitation Act require that federal agencies' electronic and information technology be accessible to people with disabilities. For information see: http://www.usability.net.

IHC accessibility and participatory design are closely linked. A Summit on eHealth and Disability held at the University of California, Berkeley in April 2001concluded that the close involvement of people with disabilities is the only way to ensure  accessibility (http://www.wellnessguide.org).

A U.S. model: IHC Designed by and for Families and People with Disabilities

One example of the application of these combined approaches is the development of IHC for several important population groups in the state of California, U.S.A. California's population of 33 million residents includes over 20 million parents and 6 million people with disabilities who have a critical need for effective IHC. The Center for Community Wellness ("Center") at the University of California, Berkeley School of Public Health coordinated a statewide health communication initiative in partnership with these groups. The goal was to involve them in the development of hard copy and online "wellness guides" that would (1) contain content relevant to their needs, (2) connect them to local services, and (3) be accessible to users with disabilities, 6th grade literacy levels, and those who read Spanish.

Methods

Developing the hard copy guides. In 1987, the Center established a participatory process through which thousands of Californians, especially those in underserved groups, became the primary designers of the guides. People of all ages, health situations and socio-economic backgrounds specified the topics, messages, format, graphics, photos and community referrals for the resources. Special attention was paid to ensure that the design would be highly "usable" by people of diverse cultures, educational levels, and those with disabilities. Schwab et. al. (1992) reported the details of these methods.

The participatory process was carried out, iteratively, through hundreds of focus groups and individual reviews statewide. Experts in many fields also contributed to this process. The state's major telephone directory publishers agreed to adopt special community service listings, which would link to headings used in the guides. After the English guide was drafted, a Spanish language edition was adaptated (rather than a translated) by intensive community reviews with Californians who represented the cultural and dialectical diversity of Spanish-speakers. After three years of development, this participatory process resulted in the production of the Wellness Guide and La Guía del Bienestar, 80-page health resources covering diverse life issues and links to local services (University of California, 1990). The guides have now been distributed to over three million families with highly positive results. Recipients significantly improved their health knowledge and confidence and attributed many positive behavior changes to using the Guide, as compared with non-recipients. (Neuhauser et al., 1998; Alcalay & Bell, 1996).

Developing content for the online guide. In 1997, the Center decided to begin the participatory development of both a print and an online guide for California's 20 million families. The guides were developed in English and Spanish using the aforementioned collaborative approach with hundreds of families and parenting experts statewide.  In this manner, users designed the content for both the published guides and the online site. As reported elsewhere, one of the challenges is to find representatives of the main user groups (Rip, Misa & Schot, 1995). For this reason, there was a special effort to create focus groups covering a very broad range of diversity.  For example, parents in the correctional system--who are often overlooked in health information development--provided ideas for the guides.

In the same year, disability groups proposed that they work with the Center to create a guide that would be more inclusive of health issues for Californians with disabilities. A statewide coalition of 22 groups with representation cross-disability and cross-culture directed the Center's coordination of this effort. As for the guide for families, hundreds of users with disabilities developed the content for a new "disability-inclusive" print guides and online site.  This marked a significant departure from the traditional approach in which people with disabilities are not considered "typical users" and are usually "designed-out" of the process (Tobias et al., 2000).

Designing a usable site. After one year of content development, print guides were produced and participatory design began for the online site. To ensure that the site would meet its envisioned goals for content relevance, resource referral and accessibility, we incorporated multiple usability tests into the design process using techniques from Nielsen's manual on Designing Web Usability (Nielsen, 2000) and traditional qualitative research methods, such as focus groups, interviews and direct observation. Combining the desired goals of high usability and accessibility posed special challenges:

1. To foster usability across a wide range of educational and literacy levels, content had been designed at a reading level of six years of education as tested by Flesch-Kincaid readability scores (Kincaid & Fishburne, 1973). To select a usable interface and format, we initially presented users with three optional designs, out of which a final design was created. Graphic literacy was further enhanced through an emphasis on a generous amount of "white space" around text and graphics.  Navigational sidebars were simplified to point to major elements of the content.  Photographs of "real people" of diverse cultures, rather than idealized graphics, were used to illustrate and provide relief from text blocks. 

2. To ensure that the site was highly accessible to people with disabilities, all elements of the site were designed to meet at least the "Priority 1" standards set by the Web Accessibility Initiative of the World Wide Web Consortium described earlier. All graphic elements were described in text ("alt-tagged") to be readable by a "screen reader" used by a blind or visually impaired person.  No Flash®, video, or audio elements were used. In addition, tabular listings of resources were tagged appropriately, using correct HTML markup for column headers and rows. Multiple revisions were made before the site was officially  launched. For example, testers proposed that the navigation structure be simplified and kept consistent across similar pages, and that the color scheme be improved to provide sufficient contrast for color-blind users.

3. Because California has tens of thousands of services available for the many Californians who are parents and/or have disabilities, interactive resource referral is a major challenge. To keep to our goals of "easy, relevant and accessible," we developed three discrete referral systems for the site:

             links to key toll-free numbers in California,

             links to selected Web sites relevant to the guide topics, and

             links to community service information for four pilot counties.

We selected and tested the toll-free numbers through suggestions of our collaborators and our decade of experience with numbers listed in the print guides. Web sites were selected and tested through a similar process. For four of California's 58 counties, we have designed a system to put easy-to-use community service listings from local phone books online. Users select their county and then choose services from a list limited to 40 categories, like "child care," "hospitals," etc. They are then referred to local Information and Referral agencies if they need more specialized assistance.

The site. In May 2001, as a result of this process, we and our many collaborators launched the online English and Spanish languageguides at: http://www.wellnessguide.org. Figure 1 shows a screen from the Spanish language site, which provides ideas for parents of children with disabilities and links to helpful services and Web sites.

Figure 1: Web site for the Wellness Guide

Formative and impact evaluation. The extensive needs assessment, iterative content development, and usability testing completed the formative evaluation using most of the methods suggested by Eng et  al. (1999). Results indicate that the site meets diverse user needs for content, easy use and accessibility. The site, itself, is the main product of the formative evaluation. In addition, an independent firm specialized in accessibility tested and evaluated the site. Results showed that the site met all WAI Priority 1 accessibility standards and, further, met most of Priority 2 and Priority 3 standards.

Currently, we are evaluating the impact of the site. The evaluation is being conducted online using an accessible e-form page that posts responses to a database. Users assess the site for ease of use and helpfulness by ranking attributes on a Likert-type scale and by adding free text comments. Onsite observation and interviews are also being conducted with users at family resource centers and disability organizations statewide. Because the site is dynamic, rather than static, all evaluation is considered "formative" and used to constantly improve the site.

Discussion and Conclusions

 The developments in interactive health communication hold great promise to positively impact health. While behavioral factors account for many of the fundamental determinants of health, our traditional health education efforts have not been very successful in modifying these behaviors. A key problem is that these efforts have not effectively engaged people to take control -- to examine their options and make their own informed choices for better health. Indeed, research suggests that, "taking more control" may be at the very physiological core of what makes people healthy.

IHC has the potential to change our past failures by engaging people in personalized searches, customized choices, and links to local services and social support. There are many exciting and successful applications.  However, current efforts are excluding many potential beneficiaries in our societies, especially the most vulnerable members. IHC needs to be accessible for everyone, regardless of disability, education, language, culture, and other attributes.  Further, the principles of "universal design" suggest that inclusive design is ultimately better for all users.

The weakness of our past efforts--non-engagement of the beneficiaries--compels us to be more collaborative. Participatory design can overcome multiple inadequacies of our current interfaces and online content. These techniques have been well tested in many fields over the past 30 years and are showing incipient success in IHC efforts. Although for many developers it is a big philosophical leap to change from "expert" to "collaborator," the rewards are large.

As described in the U.S. model, participatory design requires basic skills in coalition-building, qualitative research, and use of accessibility standards and usability guidelines. Advances in software applications that allow remote collaboration are opening vast opportunities for rapid, low-cost participatory design worldwide.

Author's Note: Linda Neuhauser, Department of Epidemiology and Biostatistics, School of Public Health, University of California, Berkeley, USA. The participatory model described in this paper was supported in part by grants and contracts from The California Endowment, the California Department of Social Services, and the California Wellness Foundation. I thank Professor S. Leonard Syme for manuscript review, and Jules Freedman, Mona Desai, Elena Conis, Kathy Simpson and John Banagan for assistance with literature review and manuscript editing.

References

Agency for Health Care Policy and Research. (1997). Consumer health informatics and patient decision-making. Final Report. Rockville, MD: US Department of Health and Human Services, AHCPR publication 98-N001.

Alcalay, R., & Bell, R. A. (1996). Ethnicity and health knowledge gaps: Impact of the California Wellness Guide on poor African American, Hispanic and non-Hispanic White women. Health Communication 9, 303-329.

Arnstein, S. (1969). Eight rungs on the ladder of citizen participation. In Chan, E. S. & Passett, B. A. (Eds.) Citizen Participation: A Case Book in Democracy. New Jersey, USA: Community Action Training Institute.

Assaf, A. R., Cummings, K. M., Graham, S., Mettlin, C., & Marshall, J.R. (1985). Comparison of three methods of teaching women how to perform breast self-examination. Health Education Quarterly 12, 159-272.

Barry, M.J., Fowler, F.J., Mulley, A.G., Henderson, J.V., & Wennberg, J .E. (1995). Patient reactions to a program designed to facilitate patient participation in treatment decisions for benign prostatic hyperplasia. Med Care 33, 771-782.

Baum, A. (2000). Behavioral and psychosocial interventions to modify pathophysiology and disease course. In Smedley, B., & Syme, S.L, (Eds.). (2000). Promoting health: Intervention strategies from social and behavioral research. Institute of Medicine. Washington, D.C.: National Academy Press.

Berners-Lee, T., (2000). From Internet site http://www.w3.org/WAI. Posted on September 19, 2000.

Bjerknes, G., & Bratteteig, T. (1995). User participation and democracy: A Discussion of Scandinavian research on system development. Scandinavian Journal of Information Systems, 7(1), 73-98.

Centers for Disease Control and Prevention. (1992). Estimated national spending on prevention-United States, 1998. Morbidity and Mortality Weekly Report 48(50), 1141-7.

Childrens's Partnership (March, 2000). Online content for low-income and underserved Americans: The digital divide's new frontier. Report posted at: http://childrenspartnership.org/pub/pub.html.

Collie, K., Cubranic, D., & Booth, K. S. (1998) Participatory design of a system for computer-supported distance art therapy. In PDC 98 Proceedings of the Participatory Design Conference. Cherkasky, J., Greenbaum, P., Mambrey, J., Pors, K. (Eds.) New York, NY, USA, 28 November û1 December 2000. CPSR, P.O. Box 717, Palo Alto, CA, USA 94302. http://www.cpsr.org

Computer Professionals for Social Responsibility (CPSR). (2000). Participatory design. From Internet site http://www.cpsr.org/program/workplace/PD.html. Posted October 12, 2000.

Curro, V. Lanni, R., Scipione, F., Grimaldi, V., & Mastoriacovo, P. (1997). Randomised controlled trial assessing the effectiveness of a booklet on the duration of breast feeding. Arch Dis Child 76, 500-503

Ehn, P. (1992). Scandinavian design: On participation and skill. In P. Adler and T. Winograd (Eds.), Usability, turning technologies into tools. Oxford: Oxford University Press,

Eng, T. R., Gustafson, D. H., Henderson, J., Jimison, H., & Patrick, K. (1999). Introduction to evaluation of interactive health communication applications. Am J Prev Med 16(1), 10-15.

Ferguson T. (1996). Health online: How to find health information, support groups, and self-help communities in cyberspace. Reading, MA: Addison-Wesley Publishing Company;

Freire, P. (1993). Pedagogy of the oppressed. London: Penguin Books.

Friend, J. K. (1999). Participatory action research and social change. SystemicPractice and Action Research 12(6), 655-658.

Good, M. (1992). Participatory design of a portable torque feedback device. In Proceeding of ACM Conference on human factors in computing systems. CHI 1992 (May 3-7), Monterey, CA: ACM Press, 439-446.

Greenbaum, J., & Kyng, M. (Eds.). (1991), Design at work: Cooperative design of computer systems. New Jersey: Lawrence Erlbaum Associates.

Greenfield, S., Kaplan, S., & Ware, J. Jr., (1985) Expanding patient involvement in care: Effects on patients' outcomes. Ann Intern Med 102, 520-528.

Greenberg, S., & Bohnet, R., (1991). GroupSketch: A multi-user sketchpad for geographically distributed small groups. In Proceedings of Graphic Interface '91 (Calgary, Alberta. July 1991. CIPS. 207-215.

Gustafson, D. H., Hawkins, R. P., Boberg, E. W., Bricker, E., Pingree, S., &
Chan, C. L. (1994).  The use and impact of a computer-based support system for people living with AIDS and HIV infection. Proc Annu Symp Comput Appl Med Care, 604-608.

Gustafson, D. H., Wise, M., McTavish F., Taylor, J. O., Wolberg, W., Stewart, J. et al. (1993). Development and pilot evaluation of a computer-based support system for women with breast cancer. J. Psychosocial Oncology 11, 69-93.

Health Care Financing Administration. (2000). Highlights-National health expenditures, 1998. From Internet site http://hcfa.hss.gov/stats/nhe-oact/hilites.htm. Posted January 10, 2000.

Heller, R. F., Elliott, H., Bray, A. E., & Alabaster, M. (1989). Reducing blood cholesterol levels in patients with peripheral vascular disease: Dietitian or diet fact sheet? Medical Journal of Australia 151, 566-568.

Horrigan, J.B. (2002). Getting serious online. Pew and the American Life Project. Report retrived March 3, 2002. http://pewinternet.org/reports.

Johansen, R. (1988). Groupware: Computer support for business teams. The Free Press.

Kim, P., Eng, T. R., Deering, M. J., & Maxfield, A. (1999). Published criteria for evaluating health related web sites: Review. British Medical Journal, 318, 647-649.

Kincaid, J.P., Fishburne, R.P., Rogers, R.L. & Chissom, B.S. (1975) Readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel. Memphis, Tenn: Naval Air Station.

Kirby, D., Harvey, P. D., Claussenius, D., & Novar, M. (1989) A direct mailing to teenage males about condom use: Its impact on knowledge, attitudes and sexual behavior. Family Planing Perspectives 21, 12-18. [Erratum appears in Family Planning Perspectives 21, 133].

Korpela, M., et al. (1996). Community participation in health informatics in Africa: An experiment in tripartite partnership in Ile-Ife, Nigeria. In Proceedings of the participatory Design Conference, November 1996. Cambridge MA., 25-34. CPSR, P.O. Box 717, Palo Alto, CA, USA 94302. http://www.cpsr.org. 

Krabbel, A., & Wetzel, I. (1998). The customization process for organizational package information systems: A challenge for participatory design. In PDC 98 Proceedings of the Participatory Design Conference. Chatfield, R., Kuhn, S. & Muller, M., (Eds.). Seattle, WA USA, 12-14 November 1998. CPSR, P.O. Box 717, Palo Alto, CA, USA 94302. http://www.cpsr.org. 

Kreps, G. (2002). Evaluating new health information technologies: Expanding the frontiers of health care delivery. In R. G. Bushko, (Ed.). The future of health technology (205-212). Amsterdam: OIS Press.

Labonté, R. (1986). Social Inequality and Healthy Public Policy. Health Promotion 1(3), 341-351.

Lowe,, J.N., Balanda, K.P., & Clare, G., (1998) Evaluation of antenatal smoking cessation programs for pregnant women. Australian and New Zealand Journal of Public Health 22(1), 55-9.

Marmot, M. (1994). Work and other factors influencing coronary health and sickness absence. Work and Stress 8(2), 191-201.

McCormack, M., & Forlizzi, J. (2000) Listening to user experience: Integrating technology with proactive wellness management. In PC 2000 Proceedings of the Participatory Design Conference. Cherkasky, J., Greenbaum, P., Mambrey, J., Pors, K. (Eds.) New York, NY, USA, 28 November û1 December 2000. CPSR, P.O. Box 717, Palo Alto, CA, USA 94302. http://www.cpsr.org. 

McEwan, B. J. (1998). Protective and damaging effects of stress mediators. New Eng J Med 338, 171-179.

McGinnis, J.M., & Foege, W.H. (1993). Actual causes of death in the United States. Journal of the American Medical Association 270(18), 2207-11.

McTavish, F. M., Gustafson, D. H., Owens, B. H., Wise, M. Taylor, J. O. Apantaku, F. M. et al. (1994). CHESS: An interactive computer system for women with breast cancer piloted with an under-served population. Proc Annu Symp Comput Appl Med Care, 599-603.

Morgan, M.W., Deber, R.B., Llewellyn-Thomas, H.A., Gladstone, P., Cusimano, R .J., O'Rourke, K., et al. (1997). A randomized trial of the ischemic health disease shared decision-making program: An evaluation of a decision aid. J Gen Intern Med 12.

Muller, M.J. & Kuhn, S. (Eds.). (1993). Participatory design. Communications of the ACM 30(4), 24-28.

National Cancer Institute. (1989). Making Health communication Programs Work. Bethesda, MD: National Institutes of Health. US Department of Health and Human Services;. NIH Publication 89-1493.

Neuhauser, L., Schwab, M., Syme, S. L., Bieber, M., & Obarski, S. K. (1998). Community participation in health promotion: Evaluation of the California Wellness Guide. Health Promotion International 13(3), 211-222.

Nichter, M. (1998). Participation in community health programs: Are we ready to move beyond rhetoric to consider methods and true politics of the possible? In PDC 98 Proceedings of the Participatory Design Conference. Chatfield, R., Kuhn, S. & Muller, M., (Eds.).  Seattle, WA USA, 12-14 November 1998. CPSR, P.O. Box 717, Palo Alto, CA, USA 94302. http://www.cpsr.org. 

Nicklas, T. A., Johnson, C. C., Farris, R., Rice, R., Lyon, L., & Shi, R. (1997). Development of a school-based nutrition intervention for high school students: Gimme 5. American Journal of Health Promotion 11(5), 315-22.

Nielsen, J. (2000). Designing Web usability. Indianapolis, IN, USA: New Riders Publishing,.

Powell, E. C., Tanz, R. R., Uyeda, A., Gaffney, M. B., & Sheehan, K. M. (2000) Injury prevention using pictorial information. Pediatrics 105(1), e16.

Puska, P., Tuomilehto, J., Salonen, J. et al. (1981). The North Karelia Project: Evaluation of a comprehensive community program for control of cardiovascular diseases. WHO/Europe Copenhagen.

Rip, A., Misa, T.J., & Schot, J. (Eds.), 2000. Managing technology in society: The approach of constructive technology assessment. London: Cassel Publishers, Ltd.

Robinson, T.N,, Patrick, K., Eng T. R,, & Gustafson D. (1998). [for the Science Panel on Interactive Communication and Health]. An evidence-based approach to interactive health communication: a challenge to medicine in the Information Age. JAMA 280, 1264-1269.

Schuler, D., & Namioka, A. (Eds.). (1993). Participatory design: Principles and practices. New Jersey: Lawrence Erlbaum Associates.

Schwab, M. G., Neuhauser, L., Margen, S. M., Syme, S. L., Ogar, D., Roppel, C., & Elite, A. (1992). The Wellness Guide: Towards a new model for community participation in health promotion. Health Promotion International 7, 27-36.

Science Panel on Interactive Communication and Health (SciPICH). (1999). Wired for health and well-being: The emergence of interactive health communication. Eng, T. R. & Gustafson, D. H., (Eds.), Washington, DC: U.S. Department of Health and Human Services, U.S. Government Printing Office.

Silvestri, B., & Flay, B. R. (1989). Smoking education: Comparison of practice and state-of-the-art. Journal of Preventive Medicine 18, 257-266.

Smedley, B., & Syme, S. L, (Eds.). (2000). Promoting health: Intervention strategies from social and behavioral research. Institute of Medicine. Washington, D.C.: National Academy Press.

Stark, C. (1998) Can participate in designing patients' health cards? In PDC 98 Proceedings of  the Participatory Design Conference. Chatfield, R., Kuhn, S. & Muller, M., (Eds.). Seattle, WA USA, 12-14 November 1998. CPSR, P.O. Box 717, Palo Alto, CA, USA 94302. http://www.cpsr.org. 

Stergachis, A., Newmann, W. E., Williams, K. J., & Schnell, M. M. (1990). The effect of a self-care minimal intervention for colds and flu on the use of medical services. Journal General Internal Medicine 5, 23-28.

Strecher, V. J., Kreuter, M., DenBoer, D. J., Kobrin, S., Hospers, H. J., & Skinner, C. S. (1994). The effects of computer-tailored smoking cessation messages in family practice seeing. J Fam Pract 39, 262-270.

Syme, S. L. (1990). Control and health: An epidemiological perspective. In Schaie, W., Rodin, J., & Schooler, C. (Eds.). Self-directedness: Cause and effect throughout the life course.  Hillsdale, N.J.: Earlbaum. 

Tobias, J. (2001, April). Accessible e-health technologies. Presented at E-Health and Disabilities Leadership Summit. Conference conducted by the University of California, Berkeley School of Public Health, Oakland, CA. http://www.wellnessguide.org.

Tobias, J., Burgstahler, S., & Otten, M. (2000). Consumers with disabilities as co-developers. In PC 2000 Proceedings of the Participatory Design Conference. Cherkasky, J., Greenbaum, P., Mambrey, J., Pors, K. (Eds.) New York, NY, USA, 28 November û1 December 2000. CPSR, P.O. Box 717, Palo Alto, CA, USA 94302. http://www.cpsr.org.

Ugalde, A. (1985). Ideological dimensions of community participation in Latin American health programs. Social Science and Medicine 21(4), 41-53.

University of California, Berkeley. (1990, 1993, 1994, 1995, 1997, 1998, 2000, 2001). The Wellness Guide and La Guía del Bienestar: Author.

U.S. Department of Health and Human Services. (1991). Healthy People 2000: National Health Promotion and Disease Prevention Objectives.Washington, D.C.: U.S. Government Printing Office.

U.S. Department of Health and Human Services. (Nov 2000). Healthy People 2010, 2nd Edition. Washington, D.C.: U.S. Government Printing Office.

Walpole, I.R., Watson, C., Moore, D., Goldblatt, J., & Bower, C. (1997). Evaluation of a project to enhance knowledge of hereditary diseases and management. Journal of Medical Genetics 34(10), 831-7.

World Health Organization (WHO). (1954). First meeting of the Expert Committee on the Health Education of the Public. Technical Report Series No. 89. World Health Organization,  Geneva.

World Health Organization (WHO). (1981). Global Strategy for Health for All by the Year 2000. Series No. 3. World Health Organization, Geneva.

Witte, K., Cameron, K.A., Lapinski, M.K., & Nzyuko, S., (1998). A theoretically based evaluation of HIV/AIDS prevention campaigns along the trans-Africa highway in Kenya. J. Health Commun 3(4), 345-63.


Copyright 2001 Communication Institute for Online Scholarship, Inc.

This file may not be publicly distributed or reproduced without written permission of the Communication Institute for Online Scholarship, P.O. Box 57, Rotterdam Jct., NY 12150 USA (phone: 518-887-2443).