Development and Evaluation of clinical practice guideline (CPG) in psychiatry

Development and evaluation of
clinical practice guideline in
psychiatry
Dr Diptadhi Mukherjee
LGBRIMH, Tezpur
Date: 22/02/16

Outline
• Introduction
• Development of guidelines
• Evaluation of guidelines
• Brief comparison of guidelines
• Indian scenario
• Conclusion

Introduction
• The history of Good Clinical Practice statute traces back to The
Hippocratic Oath- ethical code it is primarily known for its edict to
do no harm to the patient.
• Clinical practice guidelines are one of the foundations of efforts to
improve health care.
• The modern age of guidelines began with a 1992. Institute of
Medicine (IOM) report
• Defined guidelines as “systematically developed statements to
assist practitioner and patient decisions about appropriate health
care for specific clinical circumstances”
• Psychiatry, as a discipline, is relatively new in the field of evidence
based medicine.

• Some of the most well-known and widely accepted guidelines
are -
 APA guidelines by the American Psychiatric Association
 Texas Medication Algorithm Project Group (TMAP)
 Patient Outcomes Research Team (PORT)
 Canadian treatment guidelines by the Canadian Psychiatric
Association
 CANMAT (Canadian Network for Mood and Anxiety
Treatments)
 NICE guidelines by National Institute for Clinical Excellence
 Maudsley guidelines

Why are these guidelines needed?
• Improve the quality of health care.
• Reduce the use of unnecessary, ineffective or harmful
interventions.
• Facilitate the treatment of patients with maximum chance of
benefit, with minimum risk of harm, and at an acceptable cost.
• Principles of good decision making which takes account of patients'
preferences and values, clinicians values and experience, the best
available evidence and the availability of resources.
National Health and medical Research Council (NHMRC) 1999. A guide to the development, implementation and evaluation of clinical practice guidelines.

Development of CPG
• Six CPG development handbooks available.
1. Council of Europe
2. National Health and Medical Research Council of Australia
3. National Institute for Health and Clinical Excellence
4. New Zealand Guidelines Group
5. Scottish Intercollegiate Guideline Network
6. World Health Organization (WHO).

Development and Evaluation of clinical practice guideline (CPG) in psychiatry

• Target audiences and their use of guidelines
• Prioritizing topics for guideline development

• Guideline group composition and group process-
 Technical process (systematic reviews of relevant evidence) and Social
process (interpretation of the results of the systematic review and
development of recommendations)
 Sufficient experience Vs Cohesiveness
 Optimum size for a small group - between eight and ten people, although
groups of larger size have operated effectively
• Consumer involvement- “incorporate patients’ values, preferences,
knowledge, or perspectives in CPG recommendations”
– incorporating individual patients in guideline development groups;
– a ‘one off ’ meeting with patients;
– a series of workshops with patients;
– incorporating a consumer advocate in guideline development groups
Eccles et al. Implementation Science 2012, 7:60

• Managing conflicts of interest (COI) in guideline development-
 Financial COI is the best known type of COI and typically a
result of direct financial benefit related to topics discussed or
products recommended in guidelines or personal financial
interests
 Intellectual COI is another type of conflict that is increasingly
recognized and results from a guideline group member being
invested in her/her intellectual work

• Asserting strength of evidence- Grading of Recommendations,
Assessment, Development and Evaluation (GRADE) approaches
• 5 factors can lower the confidence in an estimate of effect and study
quality:
 Study design and execution (risk of bias)
 Inconsistency of the evidence across studies
 Indirectness of the evidence (including concepts of lack of generalizability,
transferability and external validity)
 Lack of precision of the estimate of the effect
 Publication bias.
• 3 factors can increase the quality of evidence:
 A strong or very strong association;
 A dose-effect relationship;
 All plausible residual confounding may be working to reduce the
demonstrated effect or increase the effect if no effect was observed.

• 4 Factors that influence recommendations-
 Quality of the available supporting body of evidence
 Balance between benefits and undesirable downsides
 Certainty about or variability in values and preferences of
patients
 Resource expenditure associated with the management
options

 Balance between benefits and undesirable downsides
• When the benefits of recommendation >>> downsides - the
recommendation will be strong.
• When the desirable consequences ≈ undesirable consequences - a
weaker recommendation is warranted.
• Relative risk/ hazard ratio/ odds ratio Vs absolute effects for a
specific population or situation
• Desirable and undesirable effects at baseline risk

 Uncertainty or variability of patient values and preferences
• Clinicians’ understanding of importance of particular outcomes for
patients can differ from that of the patients
• Data about patients’ preferences and values are often limited.
• Alternative management strategies- as a trade-off is always
necessary.
• Transparent explanation facilitates the interpretation of
recommendations
 Costs or resource utilization
• Costs are much more variable over time and geographic areas than
are other outcomes
• Higher costs will reduce the likelihood of a strong recommendation
in favor of a particular intervention
• In considering resource allocation- very specific about the setting to
which a recommendation applies and the perspective they took,
i.e., that of a patient, a third party payer or society as a whole.

 Wording recommendations
• Inform users of guidelines (e.g., clinicians, patients and their family
members, policy makers) about the degree of confidence by
specifying the strength of recommendations.
• Phrasing recommendations in an active voice as clear indications
what specific action should follow- ‘we recommend . . .’ Vs ‘we
suggest . . .’
• Alternatively- ‘clinicians should . . .’ Vs ‘clinicians might . . .’ or ‘we
conditionally recommend . . ..’
• Avoid ambiguous phrases- ‘clinically appropriate’ or ‘if necessary.’

 Should guideline panels make recommendations in the face of
very low-quality evidence?
• Option of not making a recommendation should be included for all
guideline panels- ‘insufficient evidence to make a recommendation’
category.
• Higher-quality evidence may never be obtained/ physicians need
guidance regardless- transparently lay out the judgments they
make.

Evaluation of clinical guidelines
• Numerous guidelines in every disciplines
• Great variability exists in the quality of clinical practice
guidelines
• Need to discriminate high-quality from lower-quality
guidelines.
• 24 appraisal instruments of practice guidelines

• The AGREE instrument-
• Based on the Cluzeau instrument
• It has been validated.
• It uses a numerical scoring scale- easier to compare scores.
• easy-to-use, and transparent instrument
• internationally developed and widely accepted,
• “It can possibly serve as a basis for an instrument to evaluate
the methodological quality of clinical pathways. “

• The AGREE Instrument is designed to assess guidelines developed by local,
regional, national or international groups or affiliated governmental
organizations. These include:
1. New guidelines
2. Existing guidelines
3. Updates of existing guidelines
• The AGREE Instrument is intended to be used by the following groups:
i) By policy makers to help them decide which guidelines could be
recommended for use in practice.
ii) By guideline developers to follow a structured and rigorous
development methodology and as a self-assessment tool to ensure
that their guidelines are sound.
iii) By health care providers who wish to undertake their own
assessment before adopting the recommendations

Structure and content of the AGREE Instrument
 AGREE consists of 23 key items organized in six domains. Each domain is
intended to capture a separate dimension of guideline quality.
• Scope and purpose (items 1-3) is concerned with the overall aim of the
guideline, the specific clinical questions and the target patient population.
• Stakeholder involvement (items 4-7) focuses on the extent to which the
guideline represents the views of its intended users.
• Rigour of development (items 8-14) relates to the process used to gather
and synthesize the evidence, the methods to formulate the
recommendations and to update them.
• Clarity and presentation (items 15-18) deals with the language and format
of the guideline.
• Applicability (items 19-21) pertains to the likely organizational, behavioral
and cost implications of applying the guideline.
• Editorial independence (items 22-23) is concerned with the independence
of the recommendations and acknowledgement of possible conflict of
interest from the guideline development group.

 Document-
• Recommend that you read the guideline and its accompanying
documentation fully before you start the appraisal.
 Number of appraisers
• We recommend that each guideline is assessed by at least two appraisers
and preferably four as this will increase the reliability of the assessment.
 Response scale
• Each item is rated on a 4-point scale ranging from 4 ‘Strongly Agree’ to 1
‘Strongly Disagree’, with two mid points: 3 ‘Agree’ and 2 ‘Disagree’.
 Comments
• There is a box for comments next to each item. You should use this box to
explain the reasons for your responses.

 Calculating domain scores
 Overall assessment
• A section for overall assessment is included at the end of the instrument.
• Contains a series of options ‘Strongly recommend’, ‘Recommend (with
provisos or alterations)’, ‘Would not recommend’ and ‘Unsure’.

SCOPE AND PURPOSE
1. The overall objective(s) of the guideline is (are) specifically described.
• Potential health impact of a guideline on society and populations of
patients- Rational prescribing of antidepressants in a cost-effective way
2. The clinical question(s) covered by the guideline is(are) specifically
described.
• Are selective serotonin reuptake inhibitors (SSRIs) more cost-effective
than tricyclic antidepressants (TCAs) in treatment of patients with
depression?
3. The patients to whom the guideline is meant to apply are specifically
described.
• A guideline on the management of depression only includes patients with
major depression, according to the DSM--5 criteria, and excludes patients
with psychotic symptoms and children.

STAKEHOLDER INVOLVEMENT
4. The guideline development group includes individuals from all the
relevant professional groups.
• Information about the composition, discipline and relevant expertise of
the guideline development group should be provided.
5. The patients’ views and preferences have been sought.
• For example, the development group could involve patients’
representatives, information could be obtained from patient interviews,
literature reviews of patients’ experiences could be considered by the
group
6. The target users of the guideline are clearly defined.
• For example, the target users for a guideline on headache may include
general practitioners, neurologists, psychiatrist, and psychologist
7. The guideline has been piloted among target users.
• For example, a guideline may have been piloted in one or several primary
care practices or hospitals. This process should be documented

RIGOUR OF DEVELOPMENT
8. Systematic methods were used to search for evidence.
• Search terms used, databases of systematic reviews
9. The criteria for selecting the evidence are clearly described.
• Including /excluding criterias
10. The methods used for formulating the recommendations are clearly
described.
• Areas of disagreement and methods of resolving them should be
specified- for example, a voting system, formal consensus techniques (e.g.
Delphi, Glaser techniques).
11. The health benefits, side effects and risks have been considered in
formulating the recommendations.
• These may include: survival, quality of life, adverse effects, and symptom
management or a discussion

12. There is an explicit link between the recommendations and the
supporting evidence.
• Recommendation should be linked with a list of references on which it is
based
13. The guideline has been externally reviewed by experts prior to its
publication.
14. A procedure for updating the guideline is provided.
• For example, a timescale has been given, or a standing panel receives
regularly updated literature searches and makes changes as required.

CLARITY AND PRESENTATION
15. The recommendations are specific and unambiguous.
• Recommendation should provide a concrete and precise description of
which management is appropriate in which situation and in what patient
group, as permitted by the body of evidence.
16. The different options for management of the condition are clearly
presented.
• For example, a recommendation on the management of depression may
contain the following alternatives:
• a. Treatment with TCA/ b. Treatment with SSRI/ c. Psychotherapy/
d. Combination of pharmacological and psychological therapy
17. Key recommendations are easily identifiable.
• For example, they can be summarized in a box, typed in bold, underlined
or presented as flow charts or algorithms.
18. The guideline is supported with tools for application.
• For example, a summary document, or a quick reference guide,
educational tools, patients’ leaflets, computer support, and should be
provided with the guideline.

APPLICABILITY
19. The potential organizational barriers in applying the recommendations
have been discussed.
• Recommendations may require changes in the current organization of care
within a service or a clinic which may be a barrier to using them in daily
practice.
20. The potential cost implications of applying the recommendations have
been considered.
• For example, there may be a need for more specialized staff, new
equipment, expensive drug treatment. These may have cost implications
for health care budgets. There should be a discussion of the potential
impact on resources in the guideline.
21. The guideline presents key review criteria for monitoring and/or audit
purposes.
• Clearly defined review criteria that are derived from the key
recommendations in the guideline- like follow up of MDD- HAM-D < 7 for 6
months

EDITORIAL INDEPENDENCE
22. The guideline is editorially independent from the funding body.
• Explicit statement that the views or interests of the funding body have not
influenced the final recommendations. If it is stated that a guideline was
developed without external funding, then should answer ‘Strongly Agree’.
23. Conflicts of interest of guideline development members have been
recorded.
• For example, this would apply to a member of the development group
whose research on the topic covered by the guideline is also funded by a
pharmaceutical company. There should be an explicit statement that all
group members have declared whether they have any conflict of interest

 Limitations of the AGREE instrument-
• Does not include criteria addressing how the guideline topic was
selected
• Does not include criteria addressing how the it will be
implemented.
• Does not assess whether systematic methods were used to
appraise the research evidence used to support guideline
recommendations
• Not possible to set thresholds for the scores to classify a clinical
practice guideline as ‘good’ or ‘bad’.

• NICE guideline had the highest methodological quality
according to AGREE and the highest scores in five out of six
domains.
• [‘One explanation might be that this guideline was developed
as part of a national policy within an established guideline
programme adequately resourced by the health authorities.’]
• Followed by the second edition of the American Psychiatric
Association (APA) guideline (total score=71)the Royal
Australian and New Zealand College of Psychiatrists guideline
total score= 62)

• Most (19) of the 24 guidelines did not include contributions from
key stakeholders such as patients or relatives.
• Only three guidelines considered health-economic effects of the
treatment options or other cost issues (AU, FI, GB1)
• 5 guidelines referred to particular cultural, ethnic or socioeconomic
issues either in diagnostic assessment or treatment planning (AU,
DK, GB1, SG, US1).
• Only 6 guidelines were reviewed externally by reviewers not
involved in the guideline development.
• Only 6 guidelines (AU, DK, FI, GB1, NO, US1) gave background
information and detailed recommendations for specific mental
health community treatment.

• Large variations in the type and frequency of psychosocial
interventions recommended.
• A majority of guidelines recommended some kind of family support
or family involvement
• Half had recommendations for psycho-educational interventions
and vocational rehabilitation. However, recommendations
concerning psychosocial interventions were generally not detailed.

Indian Psychiatric Society (IPS) guidelines
• Indian Psychiatric Society task force on clinical practice guidelines for
psychiatrists in India proposed clinical practice guidelines for psychiatrists
in India
• Based on the national workshop held at Jaipur in 2004.
• A detailed literature review, mentions elaborate list of the Indian studies
• Include specific recommendations about treatment of the acute,
continuation/stabilization, maintenance/stable phase of various mental
illnesses.
• Include several recommendations about pharmacological management,
group and individual therapy, vocational rehabilitation and specific
psychosocial interventions
• Distinct advantage of being easy to use and comprehend- clearly convey
the duration of treatment and other important issues/ algorithms given at
the end of chapters further simplify the user’s understanding/ a point-
wise listing of the guidelines
Indian J Psychiatry 49(4), Oct-Dec 2007

But……
• Influence of Indian studies on the final drafting of the guidelines is not
clear; the guidelines seem to be a direct adaptation of the Western
literature.
• Studies occur separately as a table or in the bibliography, without any
mention in the text.
• Depth of recommendations- lacking strength
• Cost and resource- guideline failed to conceptualize this important aspect
in the IPS guidelines- no mention of the cost of therapy of any of the
illnesses;
• Supposed to serve as guiding principles for the Indian psychiatrists- not
been properly reflected in the drafting of the IPS guidelines.
– Drugs or psychotherapeutic techniques which are not available in India
or are available at very few centers.
– Do not give enough emphasis to the available resources

But….
• Timely review/ update not done
• Specificity or ease of operationalizing- almost didactic in format,
making them more difficult to operationalize
• Contributor bias
• Lack a structured format- not uniform
• Ethnicity, psychosocial and cultural considerations- not done

Conclusion
• ??Unified guideline
• Scope of vast improvement in Indian scenario
• Guideline dissemination and implementation
• Specific mental health guidelines could be of considerable
importance in changing mental health treatment and
professional performance.

References
• Institute of Medicine Committee to Advise the Public Health Service on
Clinical Practice Guidelines. Clinical Practice Guidelines: Directions For A
New Program. Washington DC: National Academy Press, 1990.
• Vlayen J. et al. A systematic review of appraisal tools for clinical practice
guidelines: multiple similarities and one common deficit. International
Journal for Quality in Health Care 2005; Volume 17, Number 3: pp. 235–
242
• http://www.implementationscience.com/content/7/1/61
• Gaebel W et al. Schizophrenia practice guidelines: international survey
and comparison. British Journal Of Psychiatry ( 2 0 0 5 ) , 1 8 7, 2 4 8-2 5 5
• AGREE Collaboration. Development and validation of an international
appraisal instrument for assessing the quality of clinical practice
guidelines: the AGREE project. Qual Saf Health Care 2003; 12: 18–23.
• Goel D, Trivedi JK. Clinical practice guidelines for psychiatrists: Indian
Psychiatric Society guidelines vs. international guidelines: A critical
appraisal. Indian J Psychiatry 49(4), Oct-Dec 2007

Development and Evaluation of clinical practice guideline (CPG) in psychiatry

More Related Content

Viewers also liked (20)

Similar to Development and Evaluation of clinical practice guideline (CPG) in psychiatry (20)

Recently uploaded (20)

Development and Evaluation of clinical practice guideline (CPG) in psychiatry

Editor's Notes