Current Processes: Refining Evidence-based Recommendation Development

Table of Contents

Preface

Janelle Guirguis-Blake, M.D.;a Ned Calonge, M.D., M.P.H.;b Therese Miller, Dr.P.H.;c Albert Siu, M.D., M.S.P.H.;d Steven Teutsch, M.D., M.P.H.;e and Evelyn Whitlock, M.D., M.P.H.f

The U.S. Preventive Services Task Force (USPSTF) has developed new methods for evidence reviews and recommendation development. The most recent changes in the process include how the USPSTF solicits and prioritizes topics for review, updates evidence reviews and recommendations, and communicates with its audience.

This methods update was first published in the Annals of Internal Medicine. Select for copyright and source information.

Introduction

The U.S. Preventive Services Task Force (USPSTF) is an internationally recognized, independent panel of nonfederal experts in primary care, prevention, and research methods that makes evidence-based recommendations to guide the delivery of clinical preventive services. Convened and supported by the Agency for Healthcare Research and Quality (AHRQ), the USPSTF is charged by the U.S. Congress to review the scientific evidence for clinical preventive services and to develop evidence-based recommendations for their delivery to the health care community. The disciplines of USPSTF members include family medicine, internal medicine, geriatrics, preventive medicine, pediatric and adolescent medicine, obstetrics and gynecology, nursing, psychology and behavioral medicine, public health, and health policy.

Since its inception more than 20 years ago, the USPSTF has worked to fulfill its mission by:

  1. Evaluating the benefits and harms of preventive services in apparently healthy persons on the basis of age, sex, and known risk factors for disease.
  2. Making recommendations about which preventive services should be provided routinely in primary care practice and which should not.

The USPSTF recommendations are intended to improve both clinical practice and the health of patients. The scope of the Task Force is specific: Its recommendations address primary or secondary preventive services targeting conditions of substantial burden in the United States and are provided in primary care settings (or are available through primary care referral).

Although the main audience for USPSTF recommendations is the primary care clinician, these recommendations also have relevance for and are widely used by policymakers, managed care organizations, public and private payers, quality improvement organizations, research institutions, professional medical organizations, specialist physicians, and patients.

The USPSTF is distinct from other groups that provide recommendations for preventive services. It does not create guidelines based on expert opinion, as do many nonprofit advocacy organizations and professional groups. The Task Force does not advocate for prevention, perform decision analysis to routinely standardize the personal preferences and values of patients, consider medicolegal issues or the cost or coverage of services in making recommendations, or set clinical standards or health policy. Instead, the Task Force follows a unique and explicit methodology to develop recommendations that pass a rigorous evidence-based standard.1

Table 1 shows the Task Force's current procedures for developing recommendations. The USPSTF stands as an independent arbiter of the evidence and, as such, has set the standard for evidence-based recommendations for the delivery of clinical preventive services.

The process of making evidence-based recommendations occurs in an environment in which many stakeholders, often with competing interests, have their own preferences for or ideas about the delivery of preventive services. In such an environment, in which outside organizations maintain a keen interest in what the Task Force recommends, it is especially important for the USPSTF to maintain transparency, accountability, and consistency to ensure the independence and the integrity of their process and recommendations.

This paper is 1 in a series presenting the refinements that the USPSTF has undergone since its methodology was last published in 2001. The Task Force processes of selecting topics, synthesizing evidence, deliberating and voting on recommendations, soliciting peer review, and finalizing recommendations have evolved over time. The purpose of this refinement is to continually improve the methods of evidence-based review, to maintain transparency and objectivity, and to increase USPSTF efficiency. Table 2 summarizes the ways in which the USPSTF has refined its processes to meet these and other aims.

Types of Recommendations

New Topics

The Task Force solicits new topics for consideration from the field through a periodic notice in the Federal Register and solicitation of professional liaison organizations. Task Force members may also generate new topics for consideration.

The USPSTF first considers whether newly nominated topics are within the scope of the USPSTF (that is, a primary or secondary preventive service that is relevant to primary care and addresses a disease with a substantial health burden) and then prioritizes the topics by using specific criteria:

  1. The public health importance of the condition to be prevented (burden of suffering and expected effectiveness of preventive services to reduce that burden).
  2. The potential for the USPSTF to affect clinical practice (based on existing controversy or the belief that a gap exists between evidence and practice).

The USPSTF secondarily considers the need to balance the portfolio of topics to address diverse groups, types of conditions, and types of preventive services (for example, screening, counseling, and preventive medication). The USPSTF recommendation statement on routine use of aspirin or nonsteroidal anti-inflammatory drugs for the primary prevention of colorectal cancer, which appeared in the 6 March 2007 issue of Annals of Internal Medicine, represents a new topic nominated by the Centers for Disease Control and Prevention (CDC).2

Updated Recommendations

To efficiently utilize available resources, the Task Force has implemented new procedures to review previous topics and update recommendations for continued inclusion in the current Task Force library. To be consistent with the standards of the National Guidelines Clearinghouse™ (https://www.guideline.gov), the process of revisiting and updating a previous USPSTF recommendation begins approximately 3.5 years after that recommendation was released, or earlier if a landmark study is published that could change a current recommendation.

The USPSTF screens topics under consideration to identify emerging scientific issues and current clinical relevance, and then prioritizes them by using the criteria described above, in addition to considering the potential for new, recent evidence to change a previous recommendation. The USPSTF then recommends a targeted evidence update or a full evidence update. A full evidence update systematically examines a complete analytic framework of key questions by using recent evidence, taking into account any need to reframe the topic or focus of the recommendation since it was last considered, whereas a targeted evidence update systematically examines a subset of the key questions from the original analytic framework.

Reaffirmation Recommendations

Some clinical preventive services, such as screening for hypertension, have a strong, well-established evidence base and are a routine part of clinical practice. Because it is unlikely that new evidence will change USPSTF recommendations for such services, the USPSTF reviews the evidence for them in an expedited manner by conducting literature searches that address benefits and harms and consulting experts.

Some recommendations for clinical preventive services fall within the scope of not only the USPSTF but also other federal agencies. For example, adult and childhood immunizations are addressed by the CDC Advisory Committee on Immunization Practices (ACIP). In a few select cases, the Task Force chooses to refer to such recommendations. Although the USPSTF considers these recommendations part of its portfolio of recommended clinical preventive services, it refers clinicians to the ACIP active evidence review process and recommendations for 2 reasons: The USPSTF does not have adequate resources to keep such recommendations current, and it does not wish to duplicate the efforts of the ACIP.

Inactive Recommendations

The USPSTF considers some recommendations made in previous years (for example, those for electronic fetal monitoring, home uterine monitoring, and counseling for dental disease) to be no longer current or priority topics. These topics are regarded as "inactive" for various reasons.

  • First, the USPSTF may consider such recommendations now to be outside its scope of work.
  • Second, such recommendations may be judged to be no longer clinically relevant, because of changes in technology or clinical practice or because of new understanding of disease etiology or natural history.
  • Finally, the topic of a recommendation may be judged to have low priority because it has limited potential to influence public health burden or clinical practice.

Currently inactive recommendations are identified on the USPSTF Web site (https://www.uspreventiveservicestaskforce.org/).

Types of Evidence Reviews

The USPSTF bases its recommendations on systematic evidence reviews, which form the critical underpinnings of its deliberations and decisionmaking. The USPSTF members are intensively involved in the conceptualization, content, and interpretation of these reviews. The reviews are products of a partnership between members of the USPSTF and Evidence-based Practice Center (EPC), which conducts, synthesizes, and produces them. The process is facilitated and coordinated by the staff of the AHRQ, and in some cases, the AHRQ staff conducts targeted evidence updates.

The USPSTF now uses 4 types of reviews to support its recommendations:

  • Full evidence reviews.
  • Staged evidence reviews.
  • Targeted evidence updates.
  • Reaffirmation updates.

Recommendations for new topics are informed by full evidence reviews (which may be, in rare instances, staged evidence reviews). Updates of previous recommendations are informed by 1 of 3 types of reviews: full evidence updates, targeted evidence updates, or reaffirmation updates. These 3 updated reviews represent a new methodology for the USPSTF process. Table 3 provides descriptions and examples of the types of reviews.

Full evidence reviews begin with the development of an analytic framework of key questions followed by a comprehensive literature search. They then progress through critical evaluation, qualitative or quantitative synthesis as appropriate, and detailed documentation of methods and findings. The steps in the USPSTF full evidence review process are as follows:

  1. Creation of an analytic framework and key questions developed jointly by USPSTF members, EPC scientists, and AHRQ staff to guide the review process
  2. Determination of criteria for admissible evidence
  3. Evaluation of the evidence for internal and external validity of individual studies, study design and its relevance to key questions, consistency and coherence of the evidence, precision of the estimates of benefits and harms, and directness of the evidence to the key questions
  4. Estimation of the magnitude of benefits and harms for the preventive service in specific populations
  5. Assessment of the certainty of the evidence of the net benefit or harm for the preventive service in specific populations.

In select instances, critical gaps in the chain of evidence become apparent during the full evidence review of a new topic. The USPSTF may then request that the EPC conduct the systematic review in a staged manner. Staged reviews allow the USPSTF to determine whether it can make a recommendation on the basis of the review results, whether a full evidence review is required before it can make a recommendation, or whether another product (such as a commentary or an editorial) might be more appropriate than a recommendation statement. Each staged review is managed on a case-by-case basis, with the USPSTF determining at each stage how to proceed.

One example of a staged review was the USPSTF recommendation on screening for hereditary hemochromatosis,3 a topic for which there is an extensive literature on screening, penetrance is poorly understood (but probably low), the incremental benefit of earlier treatment is uncertain, and there are important harms to screening and treatment. For this review, the Task Force asked the EPC to report results for a limited number of key questions, and then determined whether the remaining key questions needed to be systematically reviewed in order to vote on a recommendation.

For many topics for which the USPSTF has made a previous recommendation, the USPSTF directs either the staff of the EPC or AHRQ to conduct a targeted evidence update rather than a full evidence update. The first step in a targeted update is to identify the update key questions, based on the analytic framework of the previous systematic evidence review. Update key questions are critical questions whose answers might result in the USPSTF making a different recommendation based on new evidence. The researchers conduct systematic evidence reviews for those critical questions, limiting the literature search to studies published since the prior full evidence review was finalized. In these cases, updates can be completed by using this targeted update process; the new information is evaluated by using the established systematic review methods, and results are integrated with the knowledge base from the previous evidence review. The USPSTF considers this evidence in updating its recommendations, rationale, and clinical considerations. The soon-to-be-released recommendations on screening for carotid artery stenosis and screening for skin cancer are examples of targeted evidence updates. (Further information is available at https://www.uspreventiveservicestaskforce.org).

For topics for which well-established evidence exists, the reaffirmation evidence update involved in supporting an updated recommendation is brief and includes literature searches in PubMed® and the Cochrane database, performed by AHRQ staff, on the benefits and harms of the preventive service. The primary goal of the literature search is to find new and substantial evidence that could change the previous recommendation. The literature search uses the Medical Subject Heading terms from the previous evidence review (if available) and searches for studies published since the last review (3 months before the end date of the previous search). For the literature search on benefits, the search is limited to meta-analyses, systematic reviews, and RCTs; for harms, the search includes meta-analyses, systematic reviews, RCTs, cohort and case-control studies, and large case series.

The USPSTF incorporates expert and peer review of its background documents to confirm that all relevant literature has been considered and that the evidence presented for USPSTF consideration is accurate. The evidence reviews on which USPSTF recommendations are based, as well as the proposed recommendation statements, are reviewed by a standard list of Federal agencies and professional organizations. Additional reviewers for the evidence reviews are identified by the EPC as national experts in the field and investigators of sentinel trials. The USPSTF requests that reviewers comment on the clarity, clinical usefulness, and scientific accuracy of the recommendation statement. The Task Force views its role as a decision maker engaged in a deliberative process. Throughout this process, the Task Force maintains its independence by making these decisions without outside influence by professional societies or governmental entities.

Communicating and Disseminating USPSTF Recommendations

The clarity and comprehensibility of recommendations are critical because of their widespread use. The USPSTF and AHRQ are aware that the recommendations and the letter grades used to define them may be misunderstood, and these agencies are therefore taking pains to clarify and refine them. AHRQ has conducted focus groups of clinicians (2004-2006) to solicit feedback about the readability and usability of the Task Force recommendations. Various themes emerged, including requests for simplified, succinct recommendations and an easier-to-use format (boldface type, bulleted sections, and boxes to highlight key information); recommendations of other professional organizations to easily compare with the USPSTF recommendation; and pointers to Web sites and references for additional information on the topic. The new recommendation grid and meanings for the letter grades appear in the USPSTF methods update that appears in this issue.4

The recommendations of the USPSTF are widely disseminated to professional audiences in relevant journals, such as Annals of Internal Medicine; on the USPSTF Web site (https://www.uspreventiveservicestaskforce.org/); in print through the annual Guide to Clinical Preventive Services; and in a Web-based Prevention TaskForce Tool, which is downloadable into personal digital assistant devices.

Representatives from federal organizations (such as the National Institutes of Health, the CDC, and the U.S. Food and Drug Administration), professional organizations (such as the American Medical Association, the American Academy of Family Physicians, the American Academy of Pediatrics, and the American Academy of Nurse Practitioners), and quality improvement organizations are invited to observe the Task Force meetings; partner organizations represent primary care clinicians, the primary audience for the Task Force recommendations. The roles of partner organizations are to inform the scope of the topic, provide expert review of evidence reports and recommendation statements, and assist with dissemination of USPSTF recommendations to their members.

The end users of the recommendations have the opportunity to respond to the USPSTF recommendations and their accompanying evidence reviews through editorials in peer-reviewed publications and through formal letters to the editor of peer-reviewed journals. These letters are an important source of feedback to the USPSTF.

Conclusion

The USPSTF believes that its recommendations and reviews should be used to foster communication among health care providers, patients, payers, employers, and research organizations for the development of quality improvement strategies. The USPSTF relies on partner organizations, such as professional societies, and on policymakers to use the USPSTF recommendations to improve the delivery of evidence-based preventive services and, when appropriate, to further research in areas identified by the USPSTF.

As evidence-based reviews evolve, so too will the USPSTF continue to refine and advance its methodology. With limited resources and a growing body of literature about clinical preventive services, the USPSTF must balance its rigorous scientific standards with the demand for up-to-date recommendations on a broad array of preventive services. Likewise, to maximize implementation of USPSTF recommendations, the USPSTF will continue to refine its communication strategies. Responding to the busy primary care clinician's need for clear, concise, evidence-based recommendations on the delivery of clinical preventive services is the mission of the USPSTF.

Notes

Potential Financial Conflicts of Interest

Employment: S. Teutsch (Merck & Co. Inc.); Stock ownership or options (other than mutual funds): S. Teutsch (Merck & Co. Inc.).

 Author Affiliations

  • a Dr. Guirguis-Blake: Department of Family Medicine, Tacoma Family Medicine Residency Program, University of Washington, Tacoma, WA.
  • b Dr. Calonge: Colorado Department of Public Health and Environment, Denver, CO.
  • c Dr. Miller: Center for Primary Care, Prevention & Clinical Partnerships, Agency for Healthcare Research and Quality, Rockville, MD.
  • d Dr. Siu: Brookdale Department of Geriatrics and Adult Development, Mount Sinai Medical Center, New York, NY.
  • e Dr. Teutsch: Merck & Company, Inc., West Point, PA.
  • f Dr. Whitlock: Kaiser Permanente Center for Health Research, Portland, OR.

Copyright and Source Information

This document is in the public domain within the United States.

Requests for linking or to incorporate content in electronic resources should be sent via the USPSTF contact form.

Source: Guirguis-Blake J, Calonge E, Miller T, Siu A, Teutsch S, Whitlock E. Current processes of the U.S. Preventive Services Task Force: refining evidence-based recommendation development. Ann Intern Med 2007;147:117-22.

References:

  1. Methods Work Group, Third U.S. Preventive Services Task Force. Current methods of the U.S. Preventive Services Task Force: a review of the process. Am J Prev Med 2001;20:21-35. [PMID: 11306229]
  2. U.S. Preventive Services Task Force. Routine aspirin or nonsteroidal antiinflammatory drugs for the primary prevention of colorectal cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med 2007;146:361-4.
  3. Whitlock EP, Garlitz BA, Harris EL, Beil TL, Smith PR. Screening for hereditary hemochromatosis: a systematic review for the U.S. Preventive Services Task Force. Ann Intern Med 2006;145:209-23. [PMID: 16880463]
  4. Barton MB, Miller T, Wolff T, Petitti D, LeFevre M, Sawaya G, et al. How to read the new recommendation statement: methods update from the U.S. Preventive Services Task Force. Ann Intern Med 2007;147:123-7.

Current as of: June 2007

Internet Citation: Current Processes: Refining Evidence-based Recommendation Development. U.S. Preventive Services Task Force. January 2014.