This site will look best in a browser that supports web standards, but it is accessible to any browser or Internet device. We recommend the Mozilla Firefox browser.

DISCERN ON THE INTERNET (February 1999)

He@lth Information on the Internet

Quality Criteria for Online Consumer Health Information on Treatment Choices

Deborah Charnock, Research Officer, Division of Public Health and Primary Health Care, University of Oxford and Sasha Shepperd, Lecturer, Department of General Practice, Imperial College, London.

Consumers require good quality evidence-based information so they can take an active role in decisions about their health care. Online consumer health information is growing at an unprecedented rate, but users and providers currently have no systematic way of judging its quality. The DISCERN instrument is a set of quality criteria developed for written consumer health information on treatment choices. DISCERN is a potentially useful tool for online information on treatment choices and an internet version is under development

Evidence-based patient choice and the need for good quality information

The growth of the internet coincides with increased consumer involvement in decisions about healthcare.  A key component of increasing consumer choice and participation is access to good quality information. Good quality information on treatment choices should be accurate and based on the best and most up-to-date scientific evidence. It should provide detailed explanations of the likely outcomes with and without treatment, and any areas of uncertainty should be dealt with honestly.

Providing consumers with information about treatment choices can reduce anxiety 1 and promote more effective relationships with health professionals.2,3 Consumers who participate in decisions about their treatment may have improved health outcomes, 2,4,5 and an understanding of treatment choices has been shown to have a positive effect on health status independent of participation in the decision-making process. 1,6

The internet has expanded the range of information available to consumers. However, the lack of editorial control has raised concerns about the quality of online health information. 7,8 Much of the information provided is not evidence-based, is biased or is inaccurate.9-11 Consumers can gain valuable insights and support from personal webpages and interactive forums, but their perception of risk may also be unduly influenced by poignant details from single cases. Most online information on treatment choices is not individualised,12 and it is not clear if consumers can select appropriate information when more than one site is available to address their concerns.8  Consequently, consumers using the internet may be misinformed about treatment choices because they are gathering information from multiple sources of varying quality and with no critical skills.13,14

Quality criteria for online consumer health information

The variable quality of health information available to consumers on the internet has led to calls for higher standards. Users are being encouraged to access reviewed sites and to reject information that does not provide evidence of authority and disclosure.8,9 However, reputable organisations and highly qualified individuals can provide information that is not evidence-based or is irrelevant to consumer needs.15 Consumers may also ignore or mistrust "respected" sources of health information 12,16 or may prefer to use less traditional and more interactive sources such as online support groups.17-19 Initiatives to improve the standard of online consumer health information must therefore take into account the wide range of networked information and the ways in which consumers may use it.

There is a clear need for a reliable appraisal tool for online health information on treatment choices which is both accessible to diverse groups of users and applicable to a wide range of information. Numerous guidelines and checklists for evaluating and producing internet resources are available, but few are health-specific and none have been subjected to rigorous scientific testing.16 Many recommendations focus on technological issues with little attention to the content of the information provided.20 Plans to develop an internet version of the DISCERN instrument 21 will provide internet users with a valuable tool for judging the quality of online consumer health information on treatment choices.

DISCERN Project

The DISCERN project was funded from 1996 to 1997 by the British Library and the NHS Executive Anglia and Oxford Research and Development Programme, and run jointly by the University of Oxford Division of Public Health and Primary Health Care, the Help for Health Trust and Buckinghamshire Health Authority. The aim of the DISCERN project was to develop an instrument to enable consumers and information providers to judge the quality of written consumer health information on treatment choices. DISCERN was also developed as a set of quality guidelines for authors.

DISCERN has undergone a rigorous process of development and testing. An expert panel representing a range of expertise in consumer health information generated criteria from a random sample of information for 3 medical conditions with varying degrees of evidence: myocardial infarction, endometriosis and chronic fatigue syndrome. A draft instrument, based on this analysis, was tested by the panel on a random sample of new material for the same 3 conditions. Tests of inter-rater reliability (weighted kappa) were conducted and the panel re-drafted the instrument to take account of the results of the test. The final pilot was conducted by a national sample of health information providers and self-help group members on a random sample of leaflets from major national self-help organisations. Tests of the pilot data achieved acceptable levels of reliability and validity. Consequently, DISCERN is the first standardised quality index of consumer health information.

The DISCERN instrument consists of 15 questions plus an overall quality rating. Each of the questions represents a separate quality criterion and is accompanied by hints to guide the user. The overall quality rating at the end of the instrument is an intuitive summary of answers to all the questions, and can be used to select and reject information or to highlight its weaknesses. A handbook has also been developed which acts as a user guide and training resource. The handbook provides background, general instructions, and detailed guidance for rating each of the separate DISCERN quality criteria including examples based on consumer health literature.

Due to its focus on treatment choices, DISCERN has become an important component in initiatives to increase consumer involvement in healthcare. The DISCERN instrument and handbook are currently being used in a variety of settings for selecting and producing printed consumer health information, for assessing the impact of providing evidence-based information on treatment choices and outcomes, and for training consumers and health professionals in shared decision-making skills.

DISCERN on the Internet

DISCERN is a potentially useful tool for the internet. A new project to develop an online version of DISCERN has been funded by the NHS Executive Anglia and Oxford Research and Development Programme. The aim of the project is to ensure that DISCERN is appropriate and accessible to all users and providers of online consumer health information on treatment choices. The first part of the project will involve the creation of a DISCERN website. The site will provide online access to the DISCERN instrument and handbook, and will be linked to a discussion group. The second part of the project will involve an online survey of users’ views of DISCERN and issues relevant to the development and use of good quality health information on the internet.

As the amount of information available to consumers continues to increase, DISCERN will play a key role in the global initiative to ensure that consumers are able to make informed treatment choices and to take an active role in their healthcare.

The DISCERN and instrument and handbook can be obtained from Sasha Shepperd.

References

1. Fallowfield LJ, Hall A, Maguire GP, Baum M. Psychological outcomes of different treatment policies in women with early breast cancer outside a clinical trial. BMJ 1990;301:575-80.

2. Greenfield S, Kaplan S, Ware JE, Jr. Expanding patient involvement in care. Effects on patient outcomes. Ann Intern Med 1985;102:520-8.

3. Meredith P, Emberton M, Wood C. New directions in information for patients. BMJ 1995;311:4-5.

4. Brody DS, Miller SM, Lerman CE, Smith DG, Caputo GC. Patient perception of involvement in medical care: relationship to illness attitudes and outcomes. J Gen Intern Med 1989;4:506-11.

5. Kaplan SH, Greenfield S, Ware JE, Jr. Assessing the effects of physician-patient interactions on the outcomes of chronic disease. Med Care 1989;27:S110-27.

6. Coulter A. Partnerships with patients: The pros and cons of shared clinical decision-making. J Health Serv Res Policy 1997; 2:112-21.

7. Wyatt JC. Commentary: measuring quality and impact of the World Wide Web. BMJ 1997;314:1879-81.

8. Sonnenberg FA. Health information on the Internet. Opportunities and pitfalls. Arch Intern Med 1997;157:151-2.

9. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor--Let the reader and viewer beware. JAMA 1997;277:1244-5.

10. Impicciatore P, Pandolfini C, Casella N, Bonati M. Reliability of health information for the public on the World Wide Web: systematic survey of advice on managing fever in children at home. BMJ 1997;314:1875-9.

11. Goldwein JW, Benjamin I. Internet-based medical information: time to take charge. Ann Intern Med 1995;123:152-3.

12. Culver JD, Gerr F, Frumkin H. Medical information on the Internet: a study of an electronic bulletin board. J Gen Intern Med 1997;12:466-70.

13. Challis DE, Johnson JM, Kennedy DS. Fetal anomaly, the Internet and the 'informed' patient: a dangerous combination. Ultrasound Obstet Gynecol 1996;8:68

14. Strum S. Consultation and patient information on the Internet: the patients' forum. Br J Urol 1997;80 Suppl 3:22-6.

15. McClung JH., Murray RD, Heitlinger LA. The internet as a source for current patient information. Pediatrics 1998;101:e2

<URL: http://www.pediatrics.org/cgi/content/full/101/6/e2> (Accessed 30th July 1998)

16. Jadad AR, Gagliardi A. Rating health information on the Internet: navigating to knowledge or to Babel? JAMA 1998;279:611-4.

17. Sharf B, F. Communicating breast cancer on-line: support and empowerment on the Internet. Women & Health 1997;26:65-84.

18. Widman LE, Tong DA. Requests for medical advice from patients and families to health care providers who publish on the World Wide Web. Arch Intern Med 1997;157:209-12.

19. Ferguson T. Health online and the empowered medical consumer. Jt Comm J Qual Improv 1997;23:251-7.

20. Cooke A, McNab A, Anagnostelis B. The good, the bad and the ugly: Internet review sites. <URL:http://omni.ac.uk/agec/iolim96.html> (Accessed 24th June 1998).

21. Charnock D, Shepperd S, Needham G, Gann R. DISCERN. An instrument for judging the quality of written consumer health information on treatment choices. Journal of Epidemiology and Community Health, in press.

This article was first published in He@lth Information on the Internet, a bi-monthly newsletter from the Wellcome Trust and the Royal Society of Medicine, providing information on health resources available on the internet.  The newsletter is written primarily for health professionals, but is accessible to all users of online health information.