Update on Methods: How To Read the New Recommendation Statement

Table of Contents

Preface

By Mary B. Barton, M.D., M.P.P.;a Therese Miller, Dr.Ph.;a Tracy Wolff, M.D., M.P.H.;a Diana Petitti, M.D., M.P.H.;b Michael LeFevre, M.D., M.S.P.H.;c George Sawaya, M.D.;d Barbara Yawn, M.D., M.S., M.Sc;e Janelle Guirguis-Blake, M.D.;f Ned Calonge, M.D., M.P.H.;g and Russell Harris, M.D., M.P.H.h for the U.S. Preventive Services Task Force.

The U.S. Preventive Services Task Force (USPSTF) has refined its methods of evidence review and assessment and created more usable documents in response to the needs of clinicians. The changes resulted in a revised grading system and a new format and new language for the Recommendation Statement.

This was first published in the Annals of Internal Medicine. Select for copyright and source information.

Abstract

Since 2001, the U.S. Preventive Services Task Force (USPSTF) has worked to refine its methods of evidence review and assessment and to create more usable documents in response to clinicians' needs. These changes have resulted in a revised grading system, as well as a new format and new language for the recommendation statement. This paper focuses on the changes to and the new look of the USPSTF recommendation statement. The new recommendation statement comprises 9 sections. Important changes include standardization of the format of the summary statement to specify what service is being recommended in what population; standardization of the headings in the rationale section; a change in the wording of the grade C recommendation and the I statement; and a new section, called "Other Considerations," in which salient issues related to cost-effectiveness, mandates, and other implementation issues are described.

Introduction

The U.S. Preventive Services Task Force (USPSTF) makes recommendations for primary care clinicians and practices about preventive services for asymptomatic patients. Each recommendation is based on a careful review and synthesis of the evidence and is released with an accompanying summary of the evidence reviewed, usually in a journal publication. All recommendations and complete evidence reviews are available on the USPSTF Web site at https://www.uspreventiveservicestaskforce.org/.

The USPSTF last described its methods in 2001.1 Since then, it has worked to refine its methods of evidence review and assessment and to create more usable documents in response to clinicians' needs. These changes have resulted in a revised grading system, as well as a new format and new language for the recommendation statement. Here, we focus on the changes to and the new look of the USPSTF recommendation statement. Discussions of other aspects of the methodological developments will unfold as a series of papers progresses. Another paper in this issue2 describes the processes whereby the USPSTF develops and communicates its recommendations. Future papers in this series will include a discussion of how to approach the consideration of a clinical preventive service when evidence is insufficient to make a recommendation for or against its use and an explanation of the process by which the USPSTF evaluates evidence and determines the certainty and magnitude of net benefit of a clinical preventive service.

Why Change the Recommendation Statement Now?

The medical literature has seen an explosion in the number of systematic reviews published in the past 10 years, both from groups using specific evidence-based methods (for example, the Cochrane Collaboration) and from other independent institutions. This change in the field of evidence assessment and synthesis, and the changes described in the following paragraphs, have made it advisable for the USPSTF to update its methods for the development of its recommendations.

The advancing methodology of systematic reviews draws attention to the fact that there may be important evidence from many types of studies. Although the well-conducted randomized, controlled trial often provides uniquely useful evidence,3 evidence from other types of studies is also critically important for making evidence-based recommendations.

An important development in the field of making recommendations from systematic reviews is reflected in the work of the GRADE (Grades of Recommendation Assessment, Development, and Evaluation) working group. This group comprises experts from around the world and is working to develop standard processes and language for assessing bodies of evidence and making recommendations on the basis of the evidence.

The approaches of the GRADE working group and the USPSTF have many elements in common. Both place separate attention on assessing the evidence and making a recommendation on the basis of the evidence. The GRADE approach assigns evidence "quality" at 1 of 4 levels: very low, low, moderate, and high, on the basis of specific criteria. The USPSTF assigns evidence "certainty" at 1 of 3 levels: high, moderate, and low, on the basis of 6 critical appraisal questions. The GRADE criteria are similar to the USPSTF's 6 questions. The recommendation phase for both GRADE4-6 and the USPSTF rely on a judgment of net benefits (benefits minus harms), including whether net benefits are positive, negative, or uncertain. The GRADE process more directly includes costs than the USPSTF approach, although the USPSTF does consider the time and effort of patients and providers. The GRADE working group is developing a system that will apply to many areas, including public health, diagnostic, treatment, and prevention issues, whereas the USPSTF is more narrowly focused on prevention.

A full description of the steps in the production of recommendations in the GRADE framework is not yet available, because several considerations in the GRADE recommendation phase are still under development. The USPSTF looks forward to an ongoing dialogue with the GRADE working group, with the hope of coming to consensus on a standard process and language to minimize confusion and maximize communication.

In 2004, AHRQ conducted focus groups with 23 community-based and academic primary care physicians in Washington, DC (2 groups), and San Diego, California (1 group), to assess the extent to which current USPSTF recommendations and products are understandable and useful to them. Focus group respondents suggested improvements in the format and dissemination of the Task Force's recommendations. They reported an interest in being able to choose the level of detail they accessed in recommendations and the form in which they accessed them (for example, in print or on a Web site). Using this first set of focus group findings, the Task Force pretested 3 possible new formats in 2005 in 4 focus groups held in Baltimore, Maryland. Participants provided feedback about how formatting could highlight key information. Further refinements to the draft "new" recommendation statement were reviewed in 2006 with 4 focus groups of practicing primary care clinicians in Baltimore and the metropolitan Washington, DC, area, during which participants offered the consistent message that busy practicing clinicians require efficient tools that are clear and concise, use simple language, and have a clear format. Clinicians want to be able to scan written documents quickly, identify the relevant patient population, and see what actions are recommended.

The New Recommendation Statement

Recommendation statements now comprise 9 major sections (Table 1).

Preamble

The preamble stresses that although evidence is the primary basis for USPSTF recommendations and statements about preventive services, the decisions made by clinicians for individual patients include other important considerations, such as the patient's clinical state and circumstances and personal preferences, factors that are important to consider when implementing any USPSTF recommendation.7 Likewise, the preamble states that policy decisions should consider local resources, constraints, expertise, and priorities. In addition, decisions about the screening and treatment of individuals and policy decisions should include a clear understanding of the evidence, which the USPSTF seeks to provide.

Summary of Recommendation and Evidence

The second part of the recommendation statement is the Summary of Recommendation and Evidence that also appears in this issue.8 This statement describes the recommendation and includes the letter grade. This is the "bottom line" of the USPSTF's statement.

The USPSTF will continue to assign a letter grade to signify its assessment of the level of its recommendation. The grade will be based, as before, on the USPSTF's assessment and synthesis of the overall evidence and the magnitude of net benefit (benefits minus harms). The evidence will no longer receive an overall assessment of "good," "fair," or "poor"; rather, the product of the evidence assessment and synthesis by the USPSTF will be expressed as levels of certainty. This change in terminology is intended to add precision to the description of the recommendation-making process and does not indicate a change in the process of evaluating the evidence. In brief, certainty represents the USPSTF's judgment about the overall evidence of net benefit. The Task Force's recommendation letter grades are explained in the table on the subject.

While the USPSTF continues to use the same letter grades as it used in the past, some of the wording has changed. The description of an A recommendation no longer contains the word "strongly;" therefore, the A and B recommendation language is now the same. The USPSTF intentionally wanted to emphasize the importance of offering interventions with A and B recommendations, rather than distinguishing them on the basis of the certainty and magnitude of net benefit. The wording of the grade C recommendation represents perhaps the most important change in tone. The previous grade C recommendation read: "The USPSTF makes no recommendation for or against routine provision of the service. The USPSTF found at least fair evidence that [the service] can improve health outcomes but concludes that the balance of benefits and harms is too close to justify a general recommendation." The new version will read: " The USPSTF recommends against routinely providing X service for Y population. There may be considerations that support providing the service in an individual patient."

The concept of the close balance of benefits and harms from the previous version (the italicized sentence in the preceding paragraph) is now captured by a new summary statement in the rationale section of the new recommendation statement: "There is at least moderate certainty that the net benefit is small." This change is meant to indicate that although there is evidence of a small net benefit, the USPSTF has judged that this net benefit is too small to justify routine implementation of the service in the target population.

When the USPSTF cannot estimate the magnitude of benefits or harms with any certainty, it assigns a grade of "I" to indicate that there is insufficient evidence to support a recommendation for or against provision of the service. In the new format, this grade will be associated with a statement, not a recommendation, because the USPSTF is not issuing a recommendation for the use or nonuse of the particular service. The USPSTF is aware of the conundrum faced by clinicians who must decide whether to offer a service in the face of insufficient evidence. If such services are used, clinicians and patients should understand that there is uncertainty about expected benefits and harms. A future paper in this series will discuss domains in which the Task Force plans to provide information to clinicians to inform both their conversations with patients and their decisions.

Structured Rationale

The Structured Rationale contains elements unique to each topic and provides a more detailed description of considerations specific to the particular recommendation or type of recommendation. For example, the rationale for a screening service includes information about the importance of the condition, detection of the condition, benefits and harms of early detection and treatment, and critical gaps in knowledge. (For an example, go to the recommendation statement that also appears in this issue).8 The structured rationale is a summary of the USPSTF's reasoning for its recommendation. This section concludes with a brief overall assessment of the evidence.

Clinical Considerations

The goal of the Clinical Considerations section is to provide clinicians with detailed information about how to provide or offer the preventive service within the clinical setting. This section addresses identification of the population for whom the recommendation is intended and populations for whom it is not intended; information relevant to I statements, as appropriate; and practical information on use of the service (for example, information on tests, periodicity, ages for starting or stopping the service, risk factors, shared decision making, and treatment).

Other Considerations

The Other Considerations section is a new part of the standard recommendation statement. It provides information that may assist clinicians and policymakers on cost and cost-effectiveness, resources required to implement the service, mandates, and current practice. In this section, the USPSTF will identify key gaps in the evidence and will discuss priorities for future research and research funding.

Discussion

The Discussion section describes the scope of the evidence review and provides additional detail on how the evidence of benefits and harms, and the collective judgment of the USPSTF, were combined to determine the recommendation. The USPSTF uses this section to summarize its interpretation of important individual studies or groups of studies and to indicate how the evidence justifies its recommendations. The reader should come away with a general appreciation of the evidence the USPSTF uses to support each recommendation.

Recommendations of Others

This section summarizes how other organizations and professional groups have judged the use or performance of this service.

References

The References section at the end of the recommendation statement gives only a small sample of the important literature on a topic and includes the citation of the evidence review. Readers can find a more complete list of references at the end of the evidence review.

Tables

A pair of tables describes what the Task Force recommendations mean and what implications they have for clinical practice.

The first table, "What the U.S. Preventive Services Task Force Grades Mean and Suggestions for Practice," provides definitions of the A, B, C, and D recommendations and the I statement, with suggestions for practice. This last element was added to emphasize how the USPSTF intends its recommendations to be used. The second table, "U.S. Preventive Services Task Force Levels of Certainty Regarding Net Benefit," contains a short narrative, with examples of how levels of certainty are defined by the USPSTF. These 2 tables will be published with each new recommendation from the USPSTF.

One-Page Clinical Summary

The USPSTF has created a new document specifically to meet the needs of practicing primary care clinicians. This 1-page clinical summary, appended to the recommendation statement, was developed in response to user feedback. It displays the recommendations and clinical considerations in an easy-to-grasp, tabular format. It is intended to provide immediate access to information related to the specific populations affected by the recommendations (for example, men, adults at increased risk, and pregnant women) and information to help clinicians provide the service or understand why the service is not recommended. Table 2 shows a template for this clinical summary page. (For a specific example, go to the figure for the recommendation statement for chlamydia infection screening.)

Clinicians who wish to learn more about the evidence and rationale that led the USPSTF to make its recommendation are encouraged to read the full recommendation statement and the supporting evidence synthesis.

Targeted Audiences for Recommendation Statement

Although the USPSTF makes recommendations about preventive services for primary care clinicians, it recognizes that the recommendation statements may provide useful information for a wider range of audiences. The new USPSTF recommendation statement provides a number of audiences with quick access to targeted information.

Busy clinicians in many areas of medicine should find particularly useful the 1-page clinical summary; the clinical considerations section; the electronic preventive services selector (available at https://www.epss.ahrq.gov, for use on the Web or for download to a personal digital assistant); and the pocket-sized, annual Guide to Clinical Preventive Services. The latter 2 resources are quick, easy-to-use tools in which all the current USPSTF recommendations are packaged in 1 handy source.

Researchers can look to the USPSTF recommendations to find gaps in the evidence identified. They should find useful the rationale section and the section on other considerations. The evidence syntheses, which are updates of all the evidence the USPSTF considered on a particular topic, should also be useful to researchers.

Policymakers and others concerned with coverage issues may be particularly interested in the recommendation rating (letter grade). They may also find information about insurance coverage, costs, and system needs, when available, in the other considerations section.

The USPSTF is committed to continually updating its methods and recommendations to maintain relevance to primary care practice. The new recommendation statement format provides an introduction to methodological developments that will be more fully developed in future articles in this series.

Acknowledgment

The authors thank Jan Genevro for her creativity and expertise in professional communication and invaluable contributions to the development of the 1-page summary document.

Notes

Author Affiliation

a Drs. Barton, Miller and Wolff: Center for Primary Care, Prevention & Clinical Partnerships, Agency for Healthcare Research and Quality, Rockville, MD
b Dr. Petitti: Department of Preventive Medicine, University of Southern California, Sierra Madre, CA
c Dr. LeFevre: University of Missouri School of Medicine, Columbia, MO
d Dr. Sawaya: Departments of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, CA
e Dr. Yawn: Olmsted Medical Center, Rochester, MN
f Dr. Guirgis-Blake: Department of Family Medicine, University of Washington Tacoma Family Medicine Program, Tacoma, WA
g Dr. Calonge: Colorado Department of Public Health and Environment, Denver, CO
h Dr. Harris: University of North Carolina School of Medicine, Chapel Hill, NC

Grant Support

The work of the USPSTF is supported by the Agency for Healthcare Research and Quality.

Potential Financial Conflicts of Interest

None disclosed.

Requests for Single Reprints

Reprints are available from the USPSTF Web site and in print through the Agency for Healthcare Research and Quality Publications Clearinghouse (800-358-9295).

Current author addresses are available at www.annals.org.

Copyright and Source Information

This document is in the public domain within the United States.

Source: Barton MB, Miller T, Wolff T, Petitti D, LeFevre M, Sawaya G, Yawn B, Guirguis-Blake J, Calonge N, Harris R. How to read the new recommendation statement: methods update from the U.S. Preventive Services Task Force. Ann Intern Med 2007;147:123-127.

References:

  1. Methods Work Group, Third US Preventive Services Task Force. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med. 2001;20:21-35. [PMID: 11306229]
  2. Guirguis-Blake J, Calonge N, Miller T, Siu A, Teutsch S, Whitlock E; U.S. Preventive Services Task Force. Current processes of the U.S. Preventive Services Task Force: refining evidence-based recommendation development. Ann Intern Med. 2007;147:117-22.
  3. The Cochrane Collaboration. The Cochrane Manual. 2007;2. Updated 21 February 2007. Accessed at www.cochrane.org/admin/manual.htm on 17 May 2007.
  4. GRADE Working Group. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches. The GRADE Working Group. BMC Health Serv Res. 2004;4:38. [PMID: 15615589]
  5. GRADE Working Group. Grading quality of evidence and strength of recommendations. BMJ. 2004;328:1490. [PMID: 15205295]
  6. GRADE Working Group. Systems for grading the quality of evidence and the strength of recommendations II: pilot study of a new system. BMC Health Serv Res. 2005;5:25. [PMID: 15788089]
  7. Haynes RB, Devereaux PJ, Guyatt GH. Physicians' and patients' choices in evidence based practice [Editorial]. BMJ.2002;324:1350. [PMID: 12052789]
  8. U.S. Preventive Services Task Force. Screening for chlamydial infection: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2007;147:128-33.

Current as of: July 2007


Internet Citation: Update on Methods: How To Read the New Recommendation Statement. U.S. Preventive Services Task Force. April 2019.